Rumbold was an honorary attaché at The Hague where his father was ambassador. In 1891 he passed the first of the required examinations and entered the Diplomatic Service. After a year at the Foreign Office in London he served in Cairo, Tehran, Vienna, Madrid and Munich between 1900 and 1913. He was then moved to Tokyo and to Berlin. In Berlin he took up the position of Counsellor. Rumbold was in charge of the British Embassy when the Ambassador, Sir Edward Goschen, went home on leave on 1 July. Rumbold conducted negotiations in the first four days of the ten that preceded the outbreak of the First World War. Rumbold left Berlin with the Ambassador on 5 August 1914 with crowds attacking the Embassy and their train. In 1916 he was appointed ambassador to Berne. After the First World War he was appointed ambassador to Poland in 1919, The following year he became the High Commissioner to Constantinople, during which he signed the Lausanne Treaty on behalf of the British Empire. He was then ambassador to Madrid from 1924 to 1928. Rumbold then went on to his last job, when he was appointed ambassador to Berlin in 1928. During this time Rumbold was in favour of appeasing the Brüning Government in the hope that this would stave off German nationalist parties, like Adolf Hitler's Nazi party. However, once Hitler came to power in 1933 he was deeply unsettled by the Nazi régime and produced a succession of despatches which were critical of the Nazis. On 26 April 1933 Rumbold sent to the Foreign Office his valedictory despatch, in which he gave an unvarnished view of Hitler, the Nazis and their ambitions:
starts with the assumption that man is a fighting animal; therefore the nation is a fighting unit, being a community of fighters...A country or race which ceases to fight is doomed...Pacifism is the deadliest sin...Intelligence is of secondary importance...Will and determination are of the higher worth. Only brute force can ensure survival of the race. The new Reich must gather within its fold all the scattered German elements in Europe...What Germany needs is an increase in territory... the idea that there is something reprehensible in chauvinism is entirely mistaken...the climax of education is military service educated to the maximum of aggressiveness...It is the duty of the government to implant in the people feeling of manly courage and passionate hatred...Intellectualism is undesirable...It is objectionable to preach international understanding... has spoken with derision of such delusive documents as peace-pacts and such delusive ideas as the spirit of Locarno.
Rumbold concluded by giving stark warnings for the future of international relations:
...it would be misleading to base any hopes on a return to sanity......which can only end in one way...I have the impression that the persons directing the policy of the Hitler government are not normal.
Rumbold married Etheldred Constantia Fane, younger daughter of the British diplomat Sir Edmund Douglas Veitch Fane by his wife Constantia Wood, a niece of the 3rd Earl of Lonsdale, on 18 July 1905. With his father's passing in November 1913, Horace succeeded him as 9th baronet. They had one son and two daughters, but the younger daughter died young in 1918. Lady Rumbold's only brother Henry Nevile Fane was married 1910 to the elder daughter of the 21st Baron Clinton, and the Rumbolds were thus indirectly related to the British Royal Family after 1923. Rumbold retired due to his age in June 1933. He died on 24 May 1941, aged 72, at his home in Tisbury, Wiltshire. He was succeeded in the baronetcy by his son, Anthony, who also became a distinguished diplomat.
