Scarborough was born in England. His father, Capt. Edmund Scarborough, was a barrister and graduate of Caius College, and an army captain, who emigrated to Virginia about 1621. He settled on the Eastern Shore of Virginia with his family around 1628, and represented the Accomac Shire in the Virginia General Assembly in the 1630s. A brother, Sir Charles Scarborough, remained in England, became a noted mathematician, studied medicine, and was a founding member of the Royal Society. A Royalist, he served as physician to Kings Charles II and James II after the Restoration. It is said that Scarborough's eldest son would drown as an adult in the York River on September 21, 1739, though these dates do not line up and there is no name attached to this legend.
Career
Scarborough was one of the most prominent of the early English settlers of the Accomac Shire of the Virginia Colony, now the Eastern Shore of Virginia. On April 28, 1651, Scarborough led a raid of some fifty men, on the nearby Pocomoke Indian village along the northern boundary of Accomac Shire, after convincing the settlers that the Indians planned to attack. At least one historian doubts the veracity of his story and suggested that he may have invented the story in order to raise enough men for the attack on the village. After the settlers captured some of the villagers and bound two of them in chains, the Indians massed along the border, and it was believed they were about to attack the English. In May all the men involved in the action were called to appear in court, including Ambrose Dixon, to account for their actions. Scarborough was exonerated, however, when the court found that his raid had been justified by the circumstances. Scarborough at various times served as Speaker of the House of Burgesses, on the Governor's Council, county sheriff, lawyer, planter, surveyor, firearms dealer, cattle rancher, merchant, ship owner, Accomack Justice, and militia colonel. He employed Indians to herd his livestock while at the same time selling guns to them and condemning them in the General Assembly for obtaining firearms. In the 1650s when England was at war with the Dutch, one of Scarborough's ships was seized en route to other colonies for trade. He retaliated by seizing a Prussian ship of similar size, no matter that it was not of Dutch ownership. In 1652, Scarborough sold his seven ships to William Burton of Boston. He also incited a scandal among a local parson to deflect criticism about his own lack of morals. Near the end of his career, Scarborough helped survey the border between the Eastern Shore of Maryland and the Eastern Shore of Virginia, the Calvert-Scarborough Line, moving it substantially northwards to keep his own holdings within the colony of Virginia. In 1657, Anthony Johnson’s white neighbor, Edmund Scarborough, forged a letter in which Johnson acknowledged a debt. Johnson did not contest the case. Johnson was illiterate and could not have written the letter; nevertheless, the court awarded Scarborough 100 acres of Johnson’s land to pay off his alleged "debt". A familiar tradition on the Eastern Shore holds that he once called local Indians to a great feast where he reported the Great Spirit would speak to them. The Indians dared not disobey, and when they assembled Scarborough fired on them from an artillery piece hidden nearby. This most likely took place in Northampton County in 1671. This was Scarborough's way of eliminating enemies and dispersing the local tribes, as well as consolidating his power in the two counties of Accomack and Northampton. By 1663 Col. Scarborough had become an enemy of the Quakers located in Accomack County, Virginia. After the group of Quakers moved to Maryland where they were offered more religious freedom, Col. Scarborough used his positions as commander of His Majesty's Forces on Virginia's Eastern Shore and Surveyor-General of the Virginia Colony to lead a force of men into Maryland and claim the area for Virginia. Scarborough's main adversary, Colonel Obedience Robbins, served as a foil to "King" Scarborough during the forty years that he was in power on the Eastern Shore. It is said that the two counties were finally created by the two arch rivals because they did not wish to live in the same municipality; hence, Northampton and Accomack Counties. Robbins also served as a burgess with Scarborough and sought to nullify any attempts by his crafty nemesis to cause trouble on the Eastern Shore, whether it be with local Indians or with local government. Scarborough also took a mistress named Ann Toft. Ann lived in Accomack from at least 1660 as a femme sole and was probably the wealthiest woman in Accomack due to her association with Scarborough. Ann bore three daughters during the 1660s who were probably fathered by Scarborough. Scarborough set Toft up in business at a plantation known as Gargaphia on present day Gargathy Neck in northern Accomack County. This of land was transferred from Scarborough to her in Feb. 1664 when Ann was 21. Gargaphia, as it was known, shows up on many maritime maps of the time and would have been a convenient stopping point for sailors and a good embarkation point for Scarborough's many products and crops. Ann married Daniel Jenifer soon after Scarborough's death from smallpox in 1671. Scarborough's main residence was a property in Accomack County on Occahannock Creek known as Hedra Cottage. Although the original house is gone, a later one stands in its place. When Scarborough died after 1671 he was buried there but it is thought that his gravestone was removed by friends to keep his enemies from desecrating his remains. A modern marker has been placed on the grounds.