Nelson is the son of Jeffery Nelson and grew up in Pleasantville, New Jersey. His favorite basketball player growing up was Kobe Bryant. He mainly focused on playing football before high school, and he was a big Philadelphia Eagles fan. Nelson attended St. Augustine Prep, where he was coached by Paul Rodio. Nelson stood 5'5 as a freshman. As a junior, Nelson averaged 18.5 points and 7.8 assists per game and led the team to a 27–2 record. In his senior season, Nelson hit the clinching shot and free throw in a 58–55 win against Christian Brothers Academy in the Battle by the Bay. In the Non-Public A title game, he scored 30 points in a 83–50 win against Don Bosco Preparatory High School. Nelson was a two-time Press of Atlantic Cityplayer of the year and finished his career with 1,625 points, third highest in school history. Despite averaging 20 points per game as a senior, Nelson was not highly recruited, as most of his scholarship offers came from Patriot League schools, and he was rated a one-star recruit by ESPN. He committed to American on October 5, 2015.
College career
In his freshman debut, Nelson scored 12 points in a 62–56 loss to Maryland. He posted back-to-back 24 point games against Wagner and St. Francis, averaging 14.9 points per game as a freshman. He was named to the Patriot League All-Rookie Team. As a sophomore, Nelson averaged 18.6 points, 5.1 assists and 3.6 rebounds per game, though the team finished 6-24. He was named to the Second Team All-Patriot League. Nelson averaged 19.8 points, 5.6 rebounds, 5.2 assists, and 2.4 steals per game as a junior. He was named to the First Team All-Patriot League. Nelson scored 28 points on February 15, 2020, in a 72–68 loss to Bucknell and broke the 2,000 point threshold. On February 26, Nelson scored 30 points in a 79–59 win against Lafayette and broke Russell Bowers' all-time scoring record at American. At the conclusion of the regular season, Nelson was named Patriot League Player of the Year. He averaged 18.5 points and 5.1 rebounds per game as a senior.
