He was born in 1742, and educated at Trinity College, Dublin. In 1769 he was called to the Irish bar, and he soon obtained a large practice, and won a great reputation both as a sound lawyer and an eloquent pleader. In 1776 he entered the Irish House of Commons as member for Fore, a seat which he held until 1783. In 1783 he was elected both for Killybegs and Tulsk in Roscommon, and preferred to sit for the latter borough; in 1784 and 1790 he was re-elected for Tulsk, and in 1798 he was chosen to represent Kildare Borough in the last Irish Parliament. His eloquence soon made him as great a reputation in the Irish parliament as at the Irish bar, and he was recognised as one of the leading orators in the days of Grattan and Flood. Though an eloquent speaker, Fitzgerald was not much of a statesman. He, however, supported all the motions of the radical parties, and in 1782 he made his most famous speech in proposing a certain measure of Catholic relief. In that year he married Catherine, younger daughter of the Rev. Henry Vesey, who was grandson of John Vesey, Archbishop of Tuam, and cousin of Lord Glentworth.
Fitzgerald never sought political office, but he eagerly accepted professional appointments, which helped him at the bar. He thus became in rapid succession Third Serjeant of Ireland in 1779, Second Serjeant in 1784, and Prime Serjeant in 1787. In all the debates which preceded the final abolition of the independent Irish parliament, Fitzgerald distinguished himself. He opposed the project of the Union with all his might, and he was certainly disadvantaged in his cause, for in 1799 he was dismissed from his post of Prime Serjeant to make way for St George Daly, who had been converted to the Unionist policy. The Irish bar insisted on showing their respect for him, and continued to give him the precedence in court over the Attorney-General and Solicitor-General which he had held as Prime Serjeant. When the Union Acts were carried, Fitzgerald accepted it, and he sat in Parliament for Ennis from 1802 to February 1808, when he resigned the seat to his son, William Vesey Fitzgerald. He, however, was re-elected in 1812, but again resigned in January 1813, when he finally retired from politics. James Fitzgerald died at Booterstown, near Dublin, on 20 January 1835, aged 93; the baroness had predeceased him on 3 January 1832. His youngest son, Henry Vesey-Fitzgerald, was Dean of Emly, and Dean of Kilmore from 1826 till his death, on 30 March 1860. He succeeded his eldest brother as third Lord Fitzgerald and Vesey in 1843.