Up to the time that the British Settlers arrived in Algoa Bay in 1820, the needs of the British garrison at Port Elizabeth had been served by chaplains on passing ships. By 1825 the settlement had grown to about 500 people. One of these was Reverend Francis McClelland (the same person who built No 7 Castle Hill). He was appointed Colonial Chaplain that same year and laid the foundation stone for the Collegiate Church of St Mary the Virgin was laid. The church was finally opened for worship in 1832.with the first organ being installed in 1867. At that stage the church already had a well established choir which continues today. In 1895 the building was gutted by fire but was quickly rebuilt in far grander style with money collected from people who included the Cape Prime Minister, Cecil Rhodes and President Paul Kruger of the Transvaal.