Sherpao started his political career with Pakistan Peoples Party in 1975 after taking retirement from Pakistan Army on the advice of then Prime Minister of PakistanZulfikar Ali Bhutto after his elder brother Hayat Sherpao was assassinated in a bomb blast in Peshawar. He was elected to National Assembly of Pakistanfor the first time in 1977 Pakistani general election on Pakistan Peoples Party seat. from NA-3. He boycotted 1985 Pakistani general election. He was re-elected to the National Assembly in 1988 Pakistani general election. He was elected as the Chief Minister of Khyber Pakhtunkhwa in 1988 and remained in the office from 2 December 1988 till 8 August 1990. He became the Leader of the Opposition in the Provincial Assembly of Khyber Pakhtunkhwa following 1990 Pakistani general election. He was re-elected as the Chief Minister of Khyber Pakhtunkhwa in 1994 following 1993 Pakistani general election. He remained Leader of the Opposition in the Provincial Assembly of Khyber Pakhtunkhwa from 1993 to 1997. He was re-elected to the National Assembly in 1997 Pakistani general election. He remained senior Vice-Chairman of the Pakistan Peoples Party from 1997 to 1999 and leader of Pakistan Peoples Party in the Provincial Assembly of Khyber Pakhtunkhwa from 1997 to 1999. In 1999, he developed differences with Benazir Bhutto. It was reported that differences between Sherpao and Benazir Bhutto were cropped after the defeat of PPP in 1997 Pakistani general election. Benazir Bhutto suspected that Sherpao played a role over the dismissal of her government. Sherpao denied the claims. After 1999 Pakistani coup d'état, he went into self-exile in the United Kingdom due to multiple corruption cases. Upon return to Pakistan for 2002 Pakistani general election, he was jailed on corruption charges. In 2002, he created his own faction, of Pakistan Peoples Party-Sherpao and late 2002, was elected unopposed as the chairman of his own faction, Pakistan Peoples Party-Sherpao. He was re-elected to the National Assembly in Pakistani general election, 2002. In November 2002, he was appointed as the Minister for Water and Power with the additional portfolio of Minister for Inter-Provincial Coordination in the federal cabinet. In December 2002, Sherpao was given additional charge of Kashmir Affairs and Northern Areas and State of Frontier Region. In 2004, he was appointed as Minister for Interior in the federal cabinet. After getting re-elected to the National Assembly in 2008 Pakistani general election from his home constituency, NA-8, Charsadda, he formed Qaumi Watan Party. He was re-elected to the National Assembly in 2013 Pakistani general election.
Assassinations attempts
In April 2007, he was injured in a suicide attack in Charsadda after a suicide bomber blew himself in a political rally which killed at least 22 people. It was the first attack on him. In December 2007, second assassination attempt was made when a suicide bomb blast targeted Sherpao which killed at least 57 in a mosque in Charsadda. In April 2015, Sherpao was targeted in a suicide attack in Charsadda for the third time. Sherpao survived the attack.