After joining the Department of Foreign Affairs in 1971, O'Sullivan held diplomatic appointments in Rome, Washington, D.C. and Cairo. Between 1991 and 1994 he was the Australian representative to the UNdisarmament office. From 1994 to 1996 he headed two divisions in the Department of Foreign Affairs and Trade. After serving as Deputy Chief of Mission at the Australian Embassy in Washington, O'Sullivan was appointed as the Australian Ambassador to Germany in 1999. O'Sullivan was a Deputy Secretary in the Department of Foreign Affairs and Trade between 2003 and 2004. He was appointed as Senior Advisor to Prime Minister John Howard in January 2004, taking up his appointment to head the Australian Security Intelligence Organisation in July 2005. Between 2004 and 2005 O'Sullivan was concurrently a Commissioner of the National Crime Commission. O'Sullivan's appointment as Director-General of Security was criticised by former Australian Secret Intelligence Service officer Warren Reed, who claimed that O'Sullivan failed to protect Reed's cover while serving as deputy to the Australian Ambassador in Cairo, leading to the abduction of Reed's operational assistant by Egyptian security agents and the closure of the ASIS station in Cairo. Within weeks of his appointment to lead ASIO, the series of Islamic suicide bombings of London occurred amid domestic calls for draconian counter-terrorism laws, followed some months later with the 2005 Bali bombings, leaving four Australian tourists dead. In 2006, O'Sullivan warned that terrorists could carry out attacks on hotels and restaurants in Australia. Following actions by two ASIO agents that a Supreme Court Justice stated was probably criminal and constituted kidnapping under the law, the court dismissed legal proceedings after assessing the ASIO interviews were inadmissible. O'Sullivan defended the actions by the ASIO agents, although he did admit that changes have been made to ASIO's procedures.
Role in "Oil for Food" scandal
According to evidence presented, in March 2006, to the Cole Inquiry, an Australian Royal Commission into the "oil for food" scandal, in June 2005 while serving as senior international advisor to the Prime Minister, O'Sullivan advised executives of the Australian Wheat Board against full co-operation with a United Nations inquiry into allegations that the AWB had breached UN sanctions by paying kickbacks to the regime of former Iraqi dictator Sadaam Hussein. Notes tendered to the Cole Inquiry showed that O'Sullivan told AWB executives to "eep your responses narrow technical. Do not blame US, complain about process." Prime Minister John Howard had previously ordered full co-operation with the UN inquiry. O'Sullivan was gagged from commenting further in public, while the Cole Inquiry was in progress.