Paine grew up in a musical family in Maine. His grandfather, an instrument maker, built the first pipe organ in the state of Maine and his father and uncles were all music teachers. His father carried on the family musical instrument business. One uncle was an organist. Another was a composer. In the 1850s Paine took lessons in organ and composition from Hermann Kotzschmar, completing his first composition, a string quartet, in 1855 at the age of 16. After his first organ recital in 1857, he was appointed organist of Portland's Haydn Society, and gave a series of recitals with the object of funding a trip to Europe where he hoped to further his music education. On arrival in Europe Paine studied organ with Carl August Haupt and orchestration with Wilhelm Friedrich Wieprecht in Berlin. He also toured Europe giving organ recitals for three years, establishing a reputation as an organist that would precede his return to the United States. After returning to the US and settling in Boston in 1861, he was appointed Harvard's first University organist and choirmaster. While acting in this role Paine offered free courses in music appreciation and music theory that would become the core curriculum for Harvard's newly formed academic music department and his appointment as America's first music professor. He would remain a member of the faculty of Harvard until 1905, just a year before his death. Paine's well received 1867 Berlin premiere of Mass in D would give Paine a reputation that helped him to shape the musical infrastructure of the United States. His pioneering courses in music appreciation and music theory made the curriculum of Department of Music at Harvard a model for American Departments of Music. His service as a director of The New England Conservatory of Music establish his place at the root of an instruction chain that leads from George Chadwick to Horatio Parker to Charles Ives. He was the first guest conductor of the Boston Symphony Orchestra in the final concerts of its first season, and his works were audience favorites. Paine is noted for beginning American's symphonic tradition. He is also known for writing America's first oratorio, the Centennial Hymn that opened the 1876 Centennial Exposition in Philadelphia, was a founder of American Guild of Organists, and co-editor of "Famous Composers and their Works". In 1889, Paine made one of the first musical recordings on wax cylinder with Theo Wangemann, who was experimenting with sound recording on the newly invented phonograph. John Knowles Paine was among the initial class of inductees into the American Classical Music Hall of Fame in 1998. The Grove Music Encyclopedia says of him:
... Paine served the Harvard community for 43 years. By his presence and by his serious concern with music in a liberal arts college he awakened a regard for music among many generations of Harvard men. His writings testify to his insistence upon the place of music within the liberal arts...
Paine Hall, the concert hall for Harvard's Department of Music, is named after him. A history of that building includes many references to his pioneering role in music at Harvard.
