Harshbarger served as District Attorney for 8 years. Amongst the cases his office handled he obtained the conviction of Gerald Amirault and other employees of the Fells Acres Day Care Center in Malden, Massachusetts. Though subsequently reversed by the Massachusetts Supreme Judicial Council the convictions of Violet and Cheryl Amirault were overturned in 1995, in Massachusetts Superior Court. Following the Judicial Council's reinstatement of the verdict, another Superior Court Judge, Isaac Borenstein, granted two separate motions for new trials to the accused. Borenstein held that the interrogations of children by Harshbarger's office and its designated experts, were so tainted by "grave errors" in the investigation process that they could not be used in any new trial. However, the Massachusetts Supreme Judicial Court once again ruled to send the women back to prison. The case has been criticized as a miscarriage of justice in publications such as The Wall Street Journal and The Nation. Harshbarger was elected President of the Massachusetts Association of District Attorneys and was awarded the Livingston Hall Award by the American Bar Association for Harshbarger's outstanding work in Juvenile Justice.
In 1990, he was elected Attorney General of Massachusetts, defeating incumbent James Shannon in the Democratic primary. He was re-elected in 1994 with 70 percent of the vote. Harshbarger was one of the first Attorneys General in the nation to sue the tobacco industry for manufacturing a product, cigarettes, which causes disease and death when used as designed by the industry. Harshbarger was elected President of the National Attorneys General Association.
In June, 2016 Harshbarger announced that he would be leaving the Democratic Party to join the United Independent Party to assist them in reaching the voter enrollment necessary to remain a recognized party in Massachusetts. He has since returned to the Democratic Party.
Since 2003, Harshbarger has practiced law in the private sector, focusing on corporate governance and related issues. He is currently Senior Counsel in the Boston law firm of Casner & Edwards, LLP. Prior to joining Casner & Edwards, Harshbarger was Senior Counsel at Proskauer Rose LLP in Boston, MA. Harshbarger has continued to serve the Commonwealth of Massachusetts as a private practice attorney by accepting appointments by both Republican and Democratic Governors of MA . He has chaired the Probation Reform Commission and Correctional Reform Commission. He also serves as member of the MA Supreme Judicial Court Management Advisory Board.
ACORN investigation
On September 22, 2009, the Association of Community Organizations for Reform Now announced that Harshbarger and Proskauer "would lead an independent inquiry into the organizational systems and processes surrounding the social services of the organization" pursuant to the recent allegations of corruption within that organization in the ACORN 2009 undercover videos controversy. Harshbarger issued his written report, essentially exonerating ACORN from any alleged illegal activity, in December 2009.