Fraser was educated at Eton and at Christ Church, Oxford, graduating B.A. and M.A. In 1847 he was appointed an officer in the 1st Life Guards, but retired with a captain's rank in 1852. He then set about entering parliament, and the ups and downs of his political career were rather remarkable. He was returned for Barnstaple in 1852, but the election was declared void on account of bribery, and the constituency was disfranchised for two years. At the election of 1857 Sir William, who had meantime been defeated at Harwich, was again returned at Barnstaple. He was, however, defeated in 1859, but was elected in 1863 at Ludlow. This seat he held for only two years, when he was again defeated and did not re-enter parliament until 1874, when he was returned for Kidderminster, a constituency he represented for six years, when he retired. He was a familiar figure at the Carlton Club, always ready with a copious collection of anecdotes of Wellington, Benjamin Disraeli and Napoleon III. William Fraser was a member of the Society of Dilettanti and the author of the book "Members of the Society of Dilettanti, 1736–1874, edited by Sir William Frazer", Chiswick Press.
Fraser bequeathed a large fortune to be accumulated in trust during twenty-one years for the benefit of his nephew, Sir Keith Alexander Fraser, eldest son of General James Keith Fraser, formerly colonel of the 1stLife Guards, who succeeded him in the baronetcy. By his will dated 1 December 1886, and proved in October 1898, he further bequeathed a splendid collection of Gillray's caricatures to the House of Lords, a similar collection of H.B.'s caricatures, and a unique set of portraits of former speakers to the House of Commons; the chairs of Thackeray and Dickens respectively to the Travellers' and Athenæum Club, Nelson's sword to the United Service Club, Byron's sofa to the Garrick, the manuscript of Gray's 'Elegy' to Eton College library, and the Duke of Marlborough's sword to the Scots Guards at St. James's Palace. The chief portion of Sir William Fraser's library was sold by auction by Messrs. Sotheby, 22 to 30 April 1901, and one thousand eight hundred and fifty-two lots fetched £20,334 18s., or more than twice what Fraser had given for them. The chief items were extra-illustrated books and books with autograph inscriptions by distinguished persons.
