Dr. Robert White arrived as one of the first "pioneer settlers" and physicians of Frederick County, Virginia between 1732 and 1735. White "staked out" his farm which consisted of along Hogue Creek south of the present-day unincorporated community of Hayfield, Virginia along U.S. Route 50. White named his farm "White Hall." The original structure that would be expanded upon to become "White's Fort" was erected between 1732 and 1735 to serve as White's primary residence. White's eldest son, Major Robert White, inherited the "White Hall" property following the death of his father in 1752. In 1763, White expanded the residence at "White Hall" by erecting a taller stone structure from the original dwelling and an adjacent stockade to serve as a fortification to protect White's family and other families residing in the vicinity of Great North Mountain from attacks by Native Americans during the French and Indian War. Following White's addition to the house built by his father, the buildings became known as "White's Fort." In July 1763, White received word that Native Americans were approaching Frederick County from the Cacapon River in Hampshire County and thus warned his fellow settlers in the area to seek safety within his fortification. One of the nearby residents, Owen Thomas, refused to abandon harvesting his crops and was subsequently killed by the approaching Native Americans. The following year in June 1764, White again received word that Native Americans were present in the area and invited his neighbors to seek safety at his fort. More than 20 of the settlers in the area heeded his call, but were attacked by Lenape tribesmen while en route to "White's Fort," after which most of them were killed. Others in the party were captured or escaped, one of which was a woman who survived a scalping. In his A History of the Valley of Virginia, Samuel Kercheval remarks of White, "who had a small fort around his house as an asylum for the people in the neighborhood." White later acquired an additional tract of along Hogue Creek from his brother John White on April 7, 1790. It was on this property in 1790 that White erected the brick mansion that would later be known as "Hayfield." The structures constituting "White's Fort" were destroyed by fire around 1919.