That used to be true, you could walk to the edge of the map and then it'd rerandomise everything: new villages, new lakes -- I don't think there were rivers back then yet. You'd lose your settlement and anything you weren't personally carrying.
It hasn't been true for a long time; it was changed in 3.12 when the worldmap was expanded to be much bigger than it previously was. That was back in 2010.