Born in Bracknell, Berkshire, Stacey began his career with the youth system at Reading as an eight-year-old in 2004, before signing his first professional contract with the club on 10 December 2013 until the end of 2015–16. He made his professional debut as a 90th-minute substitute for Jake Taylor in a 1–0 victory at home to Ipswich Town on 16 August 2014 and finished 2014–15 with six appearances. Stacey signed a new contract with Reading on 19 October 2015 to keep him at the club until the summer of 2019. On 23 November 2015, Stacey joined League Two club Barnet on a short-term loan until 26 December. He debuted a day later in a 4–2 defeat away to Notts County and completed the loan spell with only two appearances. On 24 March 2016, Stacey joined League Two club Carlisle Unitedon loan until the end of 2015–16. He made his debut a day later in a 0–0 draw away to Yeovil Town and scored his first goal in the following match, a 3–2 victory at home to Bristol Rovers. His second goal for Carlisle came on 19 April against Luton Town, when he scored a consolation goal in the 61st minute to make the score 2–1 and completed the loan spell with nine appearances and two goals. On 31 August 2016, Stacey joined League Two club Exeter City on loan until 23 January 2017. He debuted three days later in a 3–2 victory away to Colchester United. His loan was extended until the end of 2016–17 on 23 January 2017, having made 19 appearances for Exeter up to that point in the season. He played in both legs of the play-off semi-final victory over his former loan club Carlisle United, scoring a stoppage time winner in the second leg to make the score 6–5 on aggregate. The goal was voted Exeter's Goal of the Decade at the end of 2019. Stacey started in the 2017 EFL League Two play-off Final at Wembley Stadium on 28 May 2017, in which Exeter lost 2–1 to Blackpool. He completed the loan spell with 38 appearances and one goal.
Luton Town
On 26 June 2017, Stacey signed a two-year contract with League Two club Luton Town for an undisclosed fee. His contract was extended by a further year at the end of the 2017–18 season after a promotion clause was triggered as a result of Luton's promotion to League One.
AFC Bournemouth
Stacey signed for Premier League club AFC Bournemouth on 8 July 2019 on a four-year contract for an undisclosed fee.
