Gwynne was the son of the Rev. William Gwynne rector of St Michael's, Lewes, and of Denton; and was born at Lewes, Sussex, England. He was educated at St Anne's Grammar School and under the Rev. George Evans at Sheffield. He studied law, was articled with attorney Charles Willis, and then practised himself as an attorney until 1837.
Career in Australia
At the end of 1837 Gwynne was appointed clerk of court by the Supreme Court of South Australia judge John Jeffcott. Gwynne left for South Australia, arriving in Adelaide aboard the Lord Goderich on 15 April 1838. His appointment as clerk of court was not confirmed and Gwynne immediately applied for admission to the bar and practised as a barrister. In 1840 he entered into partnership with William Bartley, and later was joined by Charles Mann. Gwynne established a reputation as a lawyer, especially for his knowledge of equity law and the law of property. In 1851 he was nominated to the South Australian Legislative Council, and soon afterwards brought in a bill to establish state aid to religion, which was however defeated. In 1853, during the discussion of the proposed new constitution, he spoke in favour of a nominee upper house, but it was eventually decided that the house should be an elected one with a property qualification for voters. Gwynne was defeated at the election for the council for the seat of East Torrens in 1854, but was elected unopposed to the new Legislative Council in 1857. Gwynne opposed the Robert Torrensreal property bill, being afraid that it would have dangerous consequences. Though his opposition was not successful his criticisms had the effect of improving the bill. He was Attorney-General of South Australia in the John Baker ministry which lasted for only 10 days in August 1857, and in 1859 was appointed third judge of the South Australian Supreme Court. In 1867 he became second judge and primary judge in equity. From December 1872 to June 1873 he was acting chief justice, and in February 1877 received extended leave of absence to visit England. Gwynne was never appointed Chief Justice in spite of his seniority; he retired on a pension on 28 February 1881.
In retirement Gwynne grew oranges on a comparatively large scale, and also gave some attention to viticulture; he also bred horses. Gwynne died on 10 June 1888. He married Marian who survived him with four sons and four daughters. Gwynne was an important figure during his comparatively short career in parliament. As a lawyer he was a good pleader, and as a judge he was distinguished for his clearness of apprehension, breadth of view, strict impartiality, and excellent knowledge of the law. Sir John Downer, who had appeared before him as a young advocate, spoke of him many years later as "a very great judge". Mount Gwynne in the Northern Territory was named after Gwynne by John McDouall Stuart in 1860.