Coldharbour is habitable by mortals to some degree, as a city controlled by Meridia is located in the realm after her efforts to relocate a Tamrielic city to Coldharbour due to the boundaries between Mundus and Oblivion being weakened and in the mid-Second Era, mortals from Tamriel actually lived there and the city functioned as a normal Tamrielic city, with merchants doing business and even living in houses.
