Rwaje has a M.A. in Theology and Development Studies from the University of Edinburgh in Scotland, and a doctorate in theology in the area of leadership from Fuller Theological Seminary in Pasadena, United States. Rwaje was ordained a deacon on July 28, 1985, and a priest on July 27, 1986. He was nominated as a residential canon at St. Stephen's Cathedral in Kigali in 1989. He was elected the first bishop of the Diocese of Byumba, in August 1991, being consecrated on November 24, 1991. He became Dean of the Province of the Episcopal Church of Rwanda in 1995. He was elected Archbishop of the Province of the Anglican Church of Rwanda and bishop of the newly created Diocese of Gasabo on September 17, 2010. His installation took place on January 23, 2011, at the Kigali University Stadium, being attended by Rwandan PresidentPaul Kagame. On April 10, 2012, Rwaje took part in a joint news conference, with several religious leaders from Christian denominations, concerning the legislation that would legalize abortion in Rwanda if the pregnancy could gravely damage the health of the women or of the unborn baby and in cases of incest, rape or forced marriage. Rwaje stated that instead of accepting abortion under these circumstances, measures should be taken to address the causes "since they are the problem and not abortion". He also stated that there are people born under these cases that have gone to become "useful citizens to the nation". He expressed his full opposition to abortion, like his predecessor, Emmanuel Kolini, also present. Rwaje participated at the GAFCON's Fellowship of Confessing Anglicans Leadership Conference that took place at St. Mark's Church in Battersea, London, England, with delegates from more than 30 countries, on 23–27 April 2012, to express is committed support for orthodox Anglicanism and the worldwide Anglican realignment movement. After the departure of the Anglican Mission in the Americas, the mission of the Anglican Church of Rwanda in the United States, in December 2011, Rwaje issued a Joint Communiqué with Archbishop Robert Duncan, of the Anglican Church in North America, on April 28, 2012, addressing the question of the future of the bishops and clergy of the church body. It was decided to create a new missionary organization in the United States, that would be the PEARUSA, officially a dual sub-jurisdiction of the Anglican Church of Rwanda and of the Anglican Church in North America, since June 2012. He attended GAFCON II, that took place in Nairobi, Kenya, from 21 to 26 October 2013.
