Carse was born in about 1819 in Edinburgh and his father is said to have been Alexander Carse, a well-known painter of Scottish scenes. It is thought that he was named after James Howe, a contemporary Scottish painter of animals. , in Oldham Carse's elder brother William was trained at the Royal Academy in London, but James was enrolled at the new Royal Scottish Academy, where his father, having returned to Scotland from London, was a founding member. Although his father had exhibited at the academy between 1827 and 1836, he was not financially successful and his father needed to apply for financial assistance in 1843. He died the same year. Carse was in London in the early 1860s, exhibiting paintings of Scotland and England. His paintings at that time include several of scenes around Bolton and Oldham. In 1866, Carse won a gold medal at the Intercolonial Exhibition in Chicago.
By 1869 Carse had visited both Australia and New Zealand, and an exhibition of his new work was shown at the Melbourne Public Library. He enjoyed commercial success, as an engraving of his drawing was included on the front cover of the IllustratedMelbourne Post later that year. An engraving of his painting of Aboriginals sitting around a fire on the shore of King George Sound was commissioned by Edwin Carton Booth. This and several of his other drawings of New Zealand and Western Australia were included in Booth's Australia Illustrated, although they were attributed to "Carr". By 1876 he had helped to found Melbourne's Victorian Academy of Art and the New South Wales Academy of Art, and he had been awarded numerous prizes and awards. In that year he was described in New South Wales as the "perhaps the best painter in the colony" and his work was selling at 30 guineas a painting. In 1880 he joined a group who left the Academy of Art to create the Art Society of New South Wales. Carse was now creating a large number of paintings but from this time they diminished both in quantity and originality as he reworked old subjects. He and his friend George Podmore's home was at Mosman Bay. Carse died in 1900 from the effects of alcoholism.