The house's alternate name comes from its first and last owners: David Deshler, who built it beginning in 1752; and Elliston P. Morris, who donated it to the National Park Service in 1948. Deshler, a merchant, bought a lot from George and Anna Bringhurst in 1751–52, and constructed a four-room summer cottage. Twenty years later he built a 3-story, 9-room addition to the front, creating one of the most elegant homes in the region. Isaac Franks, a former colonel in the Continental Army, bought the house following Deshler's 1792 death. It was he who rented it to President Washington. Later, the house was sold to Elliston and John Perot, and in 1834 to Elliston's son-in-law, Samuel B. Morris. The Morris family lived in the house for over a hundred years, before its 1948 donation to the National Park Service.
When the Yellow Fever Epidemic of 1793 struck Philadelphia, President Washington remained in the city until September, before making his regular autumn trip home to Mount Vernon. He and a small group of slaves returned in early November, but Philadelphia was under quarantine and they were rerouted to Germantown, then ten miles outside the city. He first occupied the Dove House, the headmaster's residence for Germantown Academy. He also traveled to Reading, Pennsylvania, 60 miles northwest of the city, to see if it would make a suitable emergency capital. Returning to Germantown, from November 16 to 30, he occupied the Isaac Franks house. His wife Martha, two of her grandchildren, Eleanor Parke Custis and George Washington Parke Custis, and more of their slaves and staff joined him late in the stay.
The Bringhurst House, neighboring the Germantown White House on the northwest, was originally owned by John Bringhurst, a carriage builder and inventor of the "Germantown Wagon"; in 1780 he built a carriage for George Washington. His estate consisted of in Germantown, and was eventually split up by his heirs. Today, near the current historic site, "Bringhurst Street" is a street named after him which lies on the edge of his former land. Lieutenant Colonel John Bird was "lying sick" in the Bringhurst House when the American army attacked on the morning of October 4, 1777. Bird arose from bed to lead his men, but was mortally wounded in the battle. Although a surgeon tried to treat him in Melchoir Meng's house situated on what is now a part of Vernon Park, he was carried back to the Bringhurst House, where he died. In 1973, the Bringhurst house was donated to the National Park Service from the Germantown Savings Bank in order to "assure access, light, and air for the historic structure". The Bringhurst property is currently in the process of conversion into an exhibition space and welcome center for the Germantown White House landscape.