Why do non faction towns exist?

Considering that in final game you can get hostility from other factions, letting said factions gain access to new, independent, havens, will be a bad idea.

Imagine you are full hostile with NJ, and they are constantly harrassing you, they attack your havens, steal your food, spy your research. If NJ gets new havens, it will supply them with more troops and more means of harrassing you.