The Port Elizabeth harbour achieved "port" status for the first time in 1825, long before a proper harbour even existed. Back then a harbour master was appointed to regulate and oversee the offloading of ships anchored offshore with goods and people being brought to shore in rowboats. An official surfboat service was established in 1836 and this was followed by the construction of the first jetty in 1837. It wasn't until 1933 and the construction of the Charl Malan Quay (No.1 Quay, now used as the Container and Car Terminals) that Port Elizabeth had a proper port.
Due to security one can't explore the harbour properly, but you can get to the harbour wall at the bottom end of Kings Beach. Just remember that you're not allowed to walk onto the harbour wall because if you do you're going to have a security guard on your case very quickly. The view back along Kings Beach with the beachfront in the background is magnificent though.