Charles was the second son of John Ellis of Jamaica, who acquired a significant amount of wealth from sugar and slavery at a number of estates, including Montpelier, Jamaica in the parish of St James, the Newry plantation in St Mary, and the Palm estate in St Thomas-in-the-Vale. When John's brother George died young, he ran his estates on behalf of his young nephew, George Rose Ellis. However, the younger George Ellis would later complain to his maternal uncle, Edward Long, about John's avarice. Across his six Jamaican estates, John owned over 1,200 slaves, and he was ranked among the top one percent of wealthy sugar planters in Jamaica. In 1782, John and his wife Elizabeth boarded a ship from Jamaica to England, but the ship was lost at sea, and Charles inherited his father's wealthy properties in Jamaica.
Career as a sugar planter
George Rose Ellis married Anne, the daughter of Sir Peter Parker, 1st Baronet, but George died without issue in 1815, and his property passed to Charles. Charles was educated at Christ Church, Oxford, and he became an absentee planter based in England. Charles inherited the Montpelier estate, while his older brother, also named John, inherited their father's properties in the parishes of St Mary and St George. John married another daughter of Parker, named Antoinette, but died heavily in debt in 1832, and his properties were acquired by Charles. When the British government emancipated the slaves in the 1830s, Charles was compensated for his liberated slaves to the tune of over £16.000.
Political career
Charles Ellis was elected to the House of Commons for Heytesbury in 1793, a seat he held until 1796, and then represented Seaford from 1796 to 1806 and from 1812 to 1826 and East Grinstead from 1807 to 1812. In 1826 he was raised to the peerage as Baron Seaford, of Seaford in the County of Sussex. In parliament, Ellis was a prominent defender of slavery in the West Indies plantations. He was for many years considered to be the head of West India Interest, the lobby of planters and merchants in the British parliament who opposed the abolitionists.
Emancipation and imported labour
In 1832, Ellis was in Jamaica during a slave rebellion led by Samuel Sharpe, the Baptist War resulted in Ellis suffering losses on his sugar plantations to the tune of about £41,000. Lord Seaford left Jamaica for Britain in the middle of 1834, just before the Apprenticeship period was implemented following the emancipation of the slaves. According to the Legacies of British Slave-Ownership at the University College London, Seaford was awarded a payment as a slave trader in the aftermath of the Slavery Abolition Act 1833 with the Slave Compensation Act 1837. The British Government took out a £15 million loan with interest from Nathan Mayer Rothschild and Moses Montefiore which was subsequently paid off by the British taxpayers. Seaford was associated with five different claims, he owned 1018 slaves in Jamaica and received a £18,124 payment at the time. Lord Seaford was not convinced that the Apprenticeship would work, and he was a great believer in encouraging white European immigration to Jamaica. To this end, he donated land from his Montpelier estate, which was used to create a village for recently arrived German immigrants, called Seaford Town, Jamaica after him.