What I did was pile up stuff I wanted to steal in a house further away from the rest. I don't know if villagers are fine with you dropping stuff on the ground, but if so, piling it up right next to the village border would help. That way you might hope they won't hang around watching you like they do when they know you have their stuff. Drop all your excess things elsewhere to minimize penalties.
Now, you can either try to sneak in, grab the stuff, and get away unnoticed, or grab it and run. Either way it's best done deep in the night.
If they notice you, you will be chased, and if they catch you, they will initiate the 'you have to pay for those goods now' dialogue. You can just hand their stuff back to them then for only a minor hit to relationship.
Best way by far of robbing villages blind is if they're right next to water. They can't chase you on a punt.
I don't know how long they stay mad, but I do know that all the other villages in the region won't talk to you either for a time.
The 'taken' marker goes away after a while, yes - I didn't count, but it was definitely over a week.