At the 1977 general election he was elected to Dáil Éireann as a Fine Gael Teachta Dála for the Dublin Ballyfermot constituency. With the party's loss of power in 1977, the new leader, Garret FitzGerald appointed Mitchell to the Party's Front Bench as spokesperson on Labour. At the 1981 general election Mitchell was elected for the Dublin West constituency and Fine Gael dramatically increased its number of seats and formed a coalition government with the Labour Party. On his appointment as Taoiseach, Garret FitzGerald caused some surprise by excluding some of the older conservative former ministers from his cabinet. Instead young liberals like Mitchell were appointed, with Mitchell receiving the high-profile post of Minister for Justice. The Fine Gael–Labour government collapsed in January 1982, but regained power in December of that year. Mitchell again was included in a FitzGerald cabinet, as Minister for Transport. He also served as Minister for Posts and Telegraphs from 1982–87. As Minister for Transport, Mitchell granted the aviation licence to the fledgling airline Ryanair on 29 November 1985. This was granted despite strong opposition by Ireland's national carrier Aer Lingus. The issue of the licence broke Aer Lingus' stranglehold on flights to London from the Republic of Ireland. Mitchell, who was seen as being on the social liberal wing of Fine Gael was however out of favour with John Bruton when he became Fine Gael leader in 1990. When Bruton formed the Rainbow Coalition in December 1994, Mitchell was not appointed to any cabinet post. Mitchell contested and won Dáil elections in 1977, 1981, February 1982, November 1982, 1987, 1989, 1992 and 1997. He also ran unsuccessfully for the European Parliament in the 1994 and 1999 elections. He also was director of elections for Austin Currie, the Fine Gael candidate, in the 1990 presidential election. In 2001, Bruton was deposed as Fine Gael leader, and replaced by Michael Noonan. Mitchell served as his deputy from 2001 to 2002.
Mitchell also chaired the OireachtasPublic Accounts Committee. Under Mitchell's chairmanship the committee began to look at allegations of corruption and wide-scale tax evasion in the banking sector, particularly regarding Deposit interest retention tax. It was established that there was a culture of encouraging tax evasion within Irish banks, which had allowed wealthy customers to set up non-resident bank accounts into which money was transferred, enabling the account holder to avoid paying DIRT. The scandal resulted Allied Irish Banks being forced to reach a settlement of €90 million with the Revenue Commissioners in respect of DIRT evasion in 2000 in addition to thousands of tax-evaders being prosecuted including the former Minister for Justice Pádraig Flynn. The Mitchell inquiry was "shocked and horrified" at the "careless and reckless" manner in which the Governor of the Central Bank of Ireland had quoted false statistics to the Public Accounts subcommittee. Mitchell received much praise for his role in exposing the scandal.
Loss of seat and death
Though regarded in politics as one of Fine Gael's "survivors", who held onto his seat amid major boundary changes, constituency changes and by attracting working class votes in a party whose appeal was primarily middle class, Mitchell lost his Dublin Central seat at the 2002 general election. That election witnessed a large scale collapse in the Fine Gael vote, with the party dropping from 54 to 31 seats in Dáil Éireann. Although Mitchell suffered from the swing against Fine Gael in Dublin, he was not aided by the fact that Inchicore, which was considered his base in the constituency had been moved to Dublin South-Central. Jim had chosen not to run in that constituency as his brother Gay was a sitting TD running for re-election for that constituency. Mitchell had earlier had a liver transplant in an attempt to beat a rare form of cancer which had cost the lives of a number of his siblings. Though the operation was successful, the cancer returned, and Mitchell ultimately died of the disease in December 2002. His former constituency colleague and rival, Bertie Ahern, described Jim Mitchell as having made an "outstanding contribution to Irish politics."