Scrubb grew up on the west end of Louisville, Kentucky. He grew up rooting for his hometown team, the Louisville Cardinals. His father described his hometown as "a tougher part of town, which some would consider the hood." Scrubb's family dynamics negatively affected his academics.
High school career
As a freshman, Scrubb attended Central High School in Louisville but was not allowed on the basketball team due to poor academic performance. He sometimes took medications because he believed he had a learning disability. In the summer after failing his freshman year, Scrubb studied to meet the minimum requirements to start his sophomore year. As a sophomore, he transferred from Central to the more esteemed Trinity High School, a prep school in Louisville, on a need-based voucher. In his first basketball season at Trinity, Scrubb occasionally practiced with the varsity team but never played in games because his coach found his work ethic lacking. In his junior year, he enrolled in an alternative academic program at Trinity through which he joined smaller classes and made progress in school. Over the summer, he also claimed to grow from to. In his junior season, Scrubb averaged 16 points per game and was named Seventh Region Player of the Year by The Courier-Journal. As a senior, he averaged 17.8 points and 7.1 rebounds per game. Scrubb repeated as Seventh Region Player of the Year and was a finalist for the Kentucky Mr. Basketball award.
College career
On April 11, 2018, Scrubb signed to play college basketball for John A. Logan College, a junior college in Carterville, Illinois. He joined a junior college team because he was academically ineligible for an NCAA Division I scholarship, although he had intentions of later transferring to a Division I program. Scrubb made his college debut on November 1, 2018, scoring 12 points in a 106–81 win over Motlow State. On December 8, he scored 25 points and a season-high 20 rebounds in a 99–69 victory over Southeastern Illinois College. Scrubb, on January 16, posted a season-best 40 points and 13 rebounds in a 105–93 win over Rend Lake College. He finished the season averaging 20.2 points, 8.9 rebounds, and 1.6 blocks per game, shooting 46 percent from three-point range. Scrubb was named National Junior College Athletic Association Region 24 Player of the Year and Great Rivers Athletic Conference Freshman of the Year. He also earned first-team NJCAA Division I All-American honors. Scrubb parlayed his freshman success at John A. Logan into offers from many NCAA Division I programs, including Louisville, Memphis, and Texas Tech. He was ranked as the number one junior college recruit in his class after his first season. On September 28, 2019, Scrubb committed to play for Louisville following an additional year at John A. Logan. On November 1, 2019, in his sophomore season opener, Scrubb scored 13 points and battled foul trouble in an upset loss to Otero Junior College. On December 6, it was announced that he had been suspended indefinitely after returning to campus late after Thanksgiving break. As a sophomore, Scrubb averaged 21.9 points, 6.8 rebounds and 2.7 assists a game. After the season, he was named the NABC NJCAA Division I Player of the Year and repeated as a first-team NJCAA Division I All-American, while being named GRAC Player of the Year. On March 25, 2020, Scrubb declared for the 2020 NBA draft while maintaining his eligibility and did not immediately sign with an agent. On April 9, he announced that he would sign with an agent and forgo his remaining college basketball eligibility.