The bug, if it as one, is a bit more elusive. The cave is not at the centre of the F6 map cave marker, in fact that marker seems to move around a bit. Moving around zoomed out sometimes gives a message 'you have found a cave' but not always, it auto zooms in but sometimes to a mountain sometimes to a cave square, twice the cave zoomed in map did not have any sort of cave but one time it did, so I marked it on the F6 map, explored it inside and started building some fences and a door. Handily the west side is an open mire with lots of water.
I sent Sami a savegame before finding the cave so he can investigate if it is a bug or not. I intend to get some screenshots of what a newly discovered cave looks like and update the wiki so others will know what to look for, though having seen one it is now obvious.