While silver can be hard to get hold of early on and is costly, the sacrifice only sacrifices a piece of the silver item, so it's good for a few attempts.
I'd try remaining right beside the ant hill, rather than move away from it, and I'd also try an ant hill away from any construction, which includes post spruces.
I'm not aware of a failure chance for this mission.
It can also be worth noting that the various spirit missions have a much longer timeout than other missions and can also be active in parallel with normal ones, so you can wait until it gets warmer and lighter if you want/need to.