He and his sister Alexandra Gardiner Creel inherited Gardiner's Island from their aunt, Sarah Diodati Gardiner, when she died in 1953. Gardiner had long-running disputes with his sister, and her daughter, Alexandra Creel Goelet. Goelet and her husband were conservationists, while Gardiner was an enthusiastic hunter. From 1953, when Gardiner and his sister inherited the Island, until 1977, the Island's operating costs had been covered by a trust set up by the aunt from whom they inherited the property. Due to their disputes, Gardiner refused to contribute to the taxes and other costs of maintaining the property–which, at that time, were more than $1 million per year. He didn't contribute for over a decade. They, in turn, went to court to bar him from visiting the property. Gardiner also inherited Sagtikos Manor, a 10-acre heritage property on Long Island. Gardiner inherited the property, which had been in his family since the 18th century, in the 1930s. Gardiner and his wife Eunice used the property as their primary residence for several years, early in their marriage. In 1963, when the Sagtikos Manor Historical Society was founded, the Gardiners stopped using it as their primary residence, let the Historical Society use part of the structure, but insisted the Historical Society reserve a suite for him. The property was placed on the National Register of Historic Places in the 1970s. In 1986 Gardiner transferred ownership of the property to the nonprofit Robert David Lion Gardiner Foundation. Today, Sagtikos Manor stands as a small museum rich in historical value to the American Revolution and its era onward.
Personal life
In 1961, Gardiner was married to Eunice Bailey Oakes, a former British model, at St. Thomas Church on Fifth Avenue in Manhattan. Eunice, who was previously married to William Pitt Oakes, was significantly younger than he was, but the couple's marriage yielded no children. In the 1980s Gardiner tried to adopt a distant relative, so he would have an heir who could continue inherit his share of the Island, and keep it in the Gardiner name. But those efforts failed, since he was looking for a relative who was already wealthy, who, nevertheless, would agree to his conditions as to how Gardiner's Island was managed, after his death. His niece, Alexandra Creel Goelet, had already inherited his sister's half of the estate, and inherited Gardiner's half upon his death through court. Gardiner and his niece were estranged. They traversed arduous litigation toward one another over the Gardiner Estate's fund and ownership of equity such as Gardiner's Island and Sagtikos Manor which had originally meant to be preserved for the family's history.