Payne was orphaned while still a young boy when his father was shot dead in a duel. Being the eldest son he inherited Sulby Hall. Sulby Hall was located southwest of Market Harborough in Northamptonshire, was constructed in 1792 for his grandfather, Rene Payne, a partner in the banking firm of Smith, Payne and Lepper, and was designed by John Soane. It was bought by the Hon. Frederick WilliamChild Villiers, son of George Child Villiers, 5th Earl of Jersey, in 1849, having been hired between 1847 & 1849 by the Duke of Montrose, and was demolished in 1949. In addition to Sulby Hall, Payne was left £300 000 in cash and securities and was appointed Sheriff of Northamptonshire for 1826. Being rendered financially independent, he pursued a life of gambling on horses and at the card-table. A year before coming of age he wagered and lost £33 000 by Jerry's 1824 victory in the St Leger. He failed to win £60 000 when Lord Lyon beat Savernake by a short head in the 1866 Derby, and lost a further £40 000 when Cremorne beat Pell Mell by a head in 1872. His own horses brought him no greater luck. His training of horses at Sulby Hall continued without interruption from the age of twenty until his death. Here he bred Pauline, the dam of Gladiator, an exceptional sire of the nineteenth century. A notable win was that of Clementina in the 1847 1000 Guineas Stakes, a filly he had acquired against his better judgement. He inherited Musket from Lord Glasgow, and always ran him in Lord Glasgow’s colours. Payne shared the horses left him by Lord Glasgow with his lifelong friend and companion, Colonel Peel and jointly they bred the 1878 Derby winnerSefton. This was to be Payne's last winner before his death, netting him £20 000. Payne was missed for his amiable disposition, his colourful personality and his sparkling wit. Racing historian "Thormanby", the actor William Willmott Dixon, remembered him as "a true English gentleman, large-hearted, high-spirited, the pink of chivalry and the soul of honour – a man of a most lovable nature".
