Elsey began working in the shoe industry as a teenager, initially at a local branch of GallenKamp Shoes in Fredericksburg, Virginia. Not long after graduating from Stafford Senior High School in 1983, he started working for Stride Rite, and where he became a regional vice president at the age of 25. From 1999 to 2004, Elsey was the president and CEO of Footwear Specialties International. Later, he managed growth and development in the occupational footwear market with Nautilus, Skidbuster, Third Watch, and Avenger. He also worked at EJ Footwear LLC, Iron Age and Lake of the Woods. In 2000, Elsey was promoted to CEO/president of Nautilus Footwear. In March 2005, Elsey was named President of Kodiak-Terra USA Inc. He left Kodiak-Terra in April 2007 to work full time on Soles4Souls.
Charity work
In 2004 Elsey founded Soles4Souls, a nonprofit organization that sends shoes to recovering communities struck by natural disasters. Soles4Souls became one of the USA's fastest growing charities, but faced criticism over claims that a $1 donation would provide a pair of shoes to someone in need, since the used shoes they received were resold to wholesalers that then sold them to small businesses in developing countries instead of directly shipping them abroad. One shoe company donated new shoes but the charity resold them and they ended up sold in discount stores in the USA. In 2010, Elsey expanded Soles4Souls to Clothes4Souls and in 2011, he added Hope4Souls. In 2010, Elsey published his book Almost Isn’t Good Enough about his experience with nonprofit organizations. After leaving Soles4Souls, he started Elsey Enterprises, a branding and marketing company. During his tenure at Soles4Souls, he was accused of misusing charity funds for personal needs by borrowing $900,000 to refinance his personal property which was ruled illegal under Tennessee law. Elsey later repaid the money in full and refuted that he had mismanaged the funds, saying that he was safeguarding the funds during the economic downturn. In 2016, Elsey Enterprises acquired Shoes With Heart, a shoe drive fundraising business based in Southern California.