This can be done by going to Globals and then updating the links for each of the footer columns.
In the above image you there are two links added. Each of a different type. You can change the type by selecting the dropdown and choosing either Internal Link or Offsite Link. When you choose Internal Link you have the option to select an existing page on the site and add an alternate title. With offsite link you paste in the link and add a title. On the right there is also a checkbox available to make the link open in a new window.