Kaye played two years of college soccer at York University between 2012 and 2013. During his college career he would play in 29 matches and score 18 goals. He would named OUArookie of the year after his Freshman season and to the CIS all-Canadian second-team and the OUA first-team as a Sophomore. York would win the OUA championship during his final season. Kaye would leave York and join TFC Academy, the academy side of Major League Soccer's Toronto FC in 2013. In 2014, he played in League1 Ontario with the senior academy team appearing in nine matches and scoring one goal.
Kaye joined the Wilmington Hammerheads of the USL Proon loan from TFC Academy in August 2014 as part of an affiliation between the two clubs. He made his professional debut on August 23, 2014 in a league match against Orange County Blues FC. The next day, Kaye tallied his first professional goal and assist as Wilmington drew 3-3 with LA Galaxy II. In total, Kaye appeared in 7 matches and tallied 2 goals and 2 assists during his loan with the Hammerheads.
On March 12, 2015 Kaye and seven other players were signed by USL side Toronto FC II ahead of their inaugural season. He made his debut against the Charleston Battery on March 21 and would appear in 22 matches without a goal.
On January 13, 2016 Kaye would sign with USL side Louisville City FC and he would make his season debut in the opening match on March 26 against Charlotte. He would go on to play in 24 of Louisville's 28 regular season matches scoring one goal with three assists. He also played in two of Louisville's three USL Cup matches going goalless. He did, however, score in the penalty shootout in the Eastern Conference Finals match against New York Red Bulls II. A match New York would go on to win.
Kaye would miss the first two matches of the 2017 season while on international duty and would make his season debut on April 8 against Richmond. He would on to play in 19 of Louisville's 32 regular season matches scoring four goals. He also played in both of Louisville's U.S. Open Cup matches; scoring once against the Tartan Devils. In the USL Cup playoffs Kaye would start all four matches as he and Louisville would go on to win the USL Cup Final against Swope Park.
Los Angeles FC
After two seasons in Louisville, Kaye would be transferred to MLS side Los Angeles FC for their inaugural season. He would make his LAFC debut against the Seattle Sounders during the 2018 season opener. On July 26, 2018, Kaye suffered a fractured foot in the El Tráficorivalry game against the LA Galaxy, with coach Bob Bradley saying he'd need surgery, with LAFC setting no timetable for his return; NBC Sports reported that the injury would sideline the player for "4-6 months".
During his time at Louisville, Kaye played as an "attacking midfielder and winger", however following his ascension into MLS, he assumed a "box-to-box central midfield role, one that he's thriving in"; he also performed this role on international duty with Canada. LAFC coach Bob Bradley said of Kaye: "He brings a little bit more range, a little bit more ability to get around the ball, a little bit of an ability to close down, win some balls in air, get into the box in both sides. All those things make him a little bit different". During his time with Toronto FC II, Kaye was frequently used at left back.
