O'Brien was born in Ireland in County Kings. He attended the common schools, then immigrated to the United States in 1861. O'Brien served as alderman of New York City in 1864 and 1866, then became Sheriff of New York County, New York in 1867. O'Brien directly contributed to the downfall of the Tweed ring of Tammany Hall by providing city financial accounts to the New York Times in 1871. James Watson, who was a county auditor in Comptroller Dick Connolly's office and who also held and recorded the ring's books, died a week after his head was smashed by a horse in a sleigh accident on January 21, 1871. Although Tweed guarded Watson's estate in the week prior to Watson's death, and although another ring member attempted to destroy Watson's records, a replacement auditor, Matthew O'Rourke, associated with the former sheriff James O'Brien, provided city financial accounts to O'Brien, who then forwarded the accounts to the New York Times. The New York Times, at that time the only Republican associated paper in the city, was then able to reinforce stories they had previously published against the ring. O'Brien served in the New York State Senate in 1872 and 1873, during which time he founded the Apollo Hall Democracy. He was an unsuccessful candidate for mayor of New York City in 1873 and an unsuccessful candidate for election in 1874 to the Forty-fourth Congress. Richard Croker was charged with the murder of John McKenna, a lieutenant of James O'Brien during a fight on election day of 1874 with O'Brien's rival political group. O'Brien was running for Congress against the Tammany-backed Abram S. Hewitt. John Kelly, the new Tammany Hall boss, attended the trial and Croker was freed after the jury was undecided. O'Brien was then elected as an Independent Democrat to the Forty-sixth Congress, but was an unsuccessful candidate for renomination in 1880. He then worked as a stock broker until his death. He died in Manhattan on March 5, 1907 at the age of 65. He was buried in Calvary Cemetery in Queens.
