Ashley was married twice. His first wife was the former Edith Louisa Sylvia Hawkes, known as Sylvia. They were married on 3 February 1927 and divorced 28 November 1934. Upon her marriage, she became Lady Ashley, while her legal married name was Ashley-Cooper. Lord Ashley shocked London society by marrying Hawkes, an English model and actress from the chorus line. They were divorced after she began an affair with American actorDouglas Fairbanks, Sr, who was named as co-respondent in the petition for divorce. After Lord Ashley's divorce, Lady Ashley went on to marry Fairbanks; Edward John Stanley, 6th Baron Sheffield; Americanactor, Clark Gable; and Prince Dimitri Djordjadze. While she was married to four other men after her divorce from Lord Ashley, she continued to use the name Lady Ashley throughout her life. Lord Ashley's second wife was the French-born Françoise Soulier, daughter of Georges Soulier of Caudebec-en-Caux, France. Lord Ashley and Soulier were married on 31 March 1937 and remained married until his death in 1947. Their two children were:
Lord Ashley was heir apparent to the Earldom of Shaftesbury during his life. On 8 November 1943, his father appointed him a deputy lieutenant of Dorset. However, at age 46, Ashley died unexpectedly of heart disease before his father. At that time, his son, another Anthony Ashley-Cooper, became heir apparent, inheriting the earldom in 1961 upon the death of his grandfather the 9th Earl of Shaftesbury.
Military service
Lord Ashley was a cadet in the Eton College contingent of the Officers' Training Corps. He was commissioned a second lieutenant in the 94th Field Brigade, Royal Artillery on 26 June 1925. On 1 May 1926, he transferred to the Royal Wiltshire Yeomanry with the same rank. Lord Ashley was promoted to lieutenant on 12 March 1929. Soon after, on 25 May 1929, he was seconded away from the regiment to serve, from 19 August, as an aide-de-camp to Sir Frederick Sykes, Governor of Bombay, and restored to the establishment on 19 April 1930. He was restored to his unit on 2 May 1931. Lord Ashley was promoted to captain on 26 June 1937, and to major on 5 March 1938. On 5 January 1940, he was removed from the Wiltshire Yeomanry and placed on the general list of Yeomanry officers. Major Lord Ashley was transferred to the Intelligence Corps on 22 July 1940, having requested to serve as a captain during World War II. He served with the Auxiliary Units, which were highly covert Resistance groups trained to engage and counteract the expected invasion of the United Kingdom by Nazi Germany. Members of the Auxiliary Unit were stationed in covert hidden bunkers scattered throughout Great Britain. While Major Lord Ashley was trained at Coleshill House near Highworth, Wiltshire, specific details regarding his assignments and operational base remain classified. On 18 December 1941, he was transferred to the Territorial Army reserve of officers for the Royal Armoured Corps, Royal Tank Regiment. He was posthumously awarded the Efficiency Decoration.