The quest is tedious, but...
1) Other than shame and not getting the reward, there is no penalty that I know of for choosing not to attempt a quest. Or failing to complete in in time.
2) 100 squirrel hides worth of reward is a fortune - particularly on top of the loot they already carry.
I've fought 4+ robber quest groups on one char before. On rare occasion it is possible to find them with luck and good tracking skill on the overland map. Beyond that, I usually try to scout out their location by stripping myself of everything heavy and valuable and then wandering back and forth over every tile in the region. I quickly circle around in particular overland tile before moving on to the next. Once I find them, I gather up a posse from nearby villages and head in to fight.