Stone was born in Plattsburgh, New York, to Ithiel V. and Sarah Stone. His family had been among the early settlers of the region, and his father owned a large estate. As a young man, he was an engineer and lumberman before the Civil War. Stone married Mary Elizabeth Marker at the First Presbyterian Church in Pittsburgh on August 14, 1862. They would have two children, a son, Richmond and a daughter, Margaret.
Stone served as a Union Army officer during the Civil War and became noted for his stubborn defense of the McPherson Farm during the Battle of Gettysburg. He first served as major of the 13thPennsylvania Reserves, a regiment that saw action at several early war battles, including Antietam. Stone returned to Pennsylvania to help recruit new regiments; he was commissioned as colonel of the newly raised 149th Pennsylvania Volunteer Infantry in 1863. He commanded a brigade in the third division of I Corps of the Army of the Potomac in the Battle of Chancellorsville but did not see serious combat. During the Gettysburg Campaign, Stone retained command of his three Pennsylvania regiments. On July 1, 1863, on the first day of the Battle of Gettysburg, his brigade, largely composed of green troops, was posted on McPherson's Ridge south of the Chambersburg Pike. Although the brigade had not seen previous combat, it was instrumental in holding back several assaults by the Confederates. Stone moved his regiments to block attacks by Colonel John Brockenborough and Brigadier GeneralJunius Daniel. His troops held until the Iron Brigade and other Federal units fell back. Stone's men were among last to withdraw from their sector. Stone was severely wounded in the hip and arm in the fighting, and he returned home to recuperate. After his return to active duty, Stone served briefly as a brigade commander in James Wadsworth's 4th Division, V Corps during Lt. Gen. Ulysses S. Grant's 1864 Overland Campaign. He was removed from command during the Battle of the Wilderness. Stone's horse fell on him on May 5, but many presumed he had been drunk on the battlefield. Stone commanded Camp Curtin, Pennsylvania, September 7, 1864-December 15, 1864 and the Alton Military Prison in Alton, Illinois, December 15, 1864-January 27, 1865. He resigned from the volunteers on January 27, 1865. On December 12, 1864, President Abraham Lincoln nominated Stone for appointment as brevetbrigadier general, U.S. Volunteers, to rank from September 7, 1864, for "gallant services during the war, and especially at Gettysburg" and the United States Senate confirmed the appointment on February 20, 1865.
Stone Avenue in the Gettysburg National Military Park is named in his honor and memory. The palm genus Roystonea is named in memory of the work he did in road building in Puerto Rico during the capture of the island.