Goulburn's inheritance included a number of sugar estates in Jamaica, Amity Hall in the parish of Vere, now Clarendon Parish being the most important. Slave labour was still being used to work the sugar plantations when he inherited the estates. Goulburn never visited Jamaica himself, due his health and political work; he relied on attorneys to manage his estates on his behalf. One attorney in particular, Thomas Samson, held the top job at the estate from 1802–1818 and earned a reputation for cruelty towards Goulburn's slaves. By 1818. the income from his Jamaican estates halved to less than £3,000, "although he did console himself that the condition of his slaves had probably improved".
Political career
In 1808, Goulburn became Member of Parliament for Horsham. In 1810, he was appointed Under-Secretary of State for Home Affairs, and two and a half years later, he was made Under-Secretary of State for War and the Colonies. It was in this capacity that James Meehan named Goulburn, New South Wales after him, a naming that was ratified by Governor Lachlan Macquarie. Still retaining office in the Tory government, he became a Privy Counsellor in 1821, and shortly afterwards was appointed Chief Secretary for Ireland, a position which he held until April 1827. Here, although he was frequently denounced as he was considered an Orangeman, he had a successful period of office on the whole, and in 1823 he managed to pass the Composition for Tithes Act 1823. In January 1828, he was made Chancellor of the Exchequer under the Duke of Wellington; like his leader, he disliked Roman Catholic emancipation, which he voted against in 1828. In the domain of finance, Goulburn's chief achievements were to reduce the rate of interest on part of the national debt, and to allow anyone to sell beer upon payment of a small annual fee, a complete change of policy with regard to the drink traffic. Leaving office with Wellington in November 1830, Goulburn was Home Secretary under Sir Robert Peel for four months in 1835, and when this statesman returned to office in September 1841 he became Chancellor of the Exchequer for the second time. Although Peel himself did some of the chancellor's work, Goulburn was responsible for a further reduction in the rate of interest on the national debt, and he aided his chief in the struggle which ended in the repeal of the Corn Laws. With his colleagues, he left office in June 1846. After representing Horsham in the House of Commons for over four years, Goulburn was successively member for St Germans, for West Looe, and for the city ofArmagh. In May 1831, he was elected for Cambridge University, and he retained this seat until his death. According to the Legacies of British Slave-Ownership at the University College London, Goulburn was awarded a payment as a slave trader in the aftermath of the Slavery Abolition Act 1833 with the Slave Compensation Act 1837. The British Government took out a £15 million loan with interest from Nathan Mayer Rothschild and Moses Montefiore which was subsequently paid off by the British taxpayers. Goulburn was associated with two different claims, he owned 277 slaves in Jamaica and received a £5,601 payment at the time. Goulburn was a member of the Canterbury Association from 27 March 1848.