Baldwin made his first-class debut for Hampshire in the 1877 season against Derbyshire. This would be Baldwin's only match for Hampshire until 1887. By 1887 Hampshire had been stripped of their first-class status and were now playing non-first-class matches. His first match on his return in 1887 was against Surrey. During Hampshire's period as a non first-class county, Baldwin played 66 matches, the last of which in the clubs non first-class period came against Essex in 1894. Baldwin was still playing for the club in 1895 when they regained first-class status. Eighteen years after his first first-class game, Baldwin played his second first-class match against Somerset. This marks Baldwin with the distinction of being one of three cricketers, the others being Russell Bencraft and Teddy Wynyard, to have played first-class cricket for Hampshire in both periods for which the club had first-class status. In the same season Baldwin represented the Players in the Gentlemen v Players match. Baldwin was one of the best county bowlers of his time, taking 580 wickets for Hampshire in 150 first-class matches for the county. His bowling average was just 24.79. He took 41 five wicket hauls and 6 ten wickets hauls. His best season was in the 1895 County Championship where Baldwin took 114 wickets at an average of 15.77 with best figures of 7–42. Baldwin took more than fifty wickets in seasonseven times. Baldwin scored 1,873 runs with the bat at an average of 10.70, with one half-century: a score of 55* in his final season in 1905. Baldwin played two additional non first-class matches for Hampshire against the Marylebone Cricket Club in 1897 and the a match against the touring West Indians in 1900. Baldwin's final first-class match came against local rivals Sussex in the 1905 County Championship, which brought to an end an illustrious county career. In many quarters Baldwin was considered unlucky not to have represented England in Test cricket. Baldwin was the first Hampshire professional to have a benefit.
Baldwin's son Herbert Baldwin represented Surrey between 1922 and 1930. Herbert like his father also stood as a first-class umpire, as well an umpiring in nine Test matches.
