I always choose attributes, though sometimes I make an archive save before finishing to be able to choose spells, for testing. But for continued play it's attributes.
Attributes generally are most important when creating the character (they govern how much initial skill will be), but some affect it all the time or during special events:
Endurance decreases penalty for encumbrance (and thus indirectly increases Mobility, and most if not all skills), it is also tested for shock (when you lose consciousness from heavy injuries). It's probably tested during swimming trouble (for drowning), though admittedly if this happens you're almost certain dead anyway.
Dexterity is tested with fumble tests (when you fight and drop your weapon it was a failed DEX test, in addition to failure in an attack/defence skill)
Agility is tested with stumble test (when you fight and trip over you failed AGL test)
Speed you mentioned, it increases Mobility directly
Touch is tested when eating for some reason (maybe how long it will take), it's also tested when performing crafting, at the start. It is actually tested when performing item related functions like shootin from bow or felling a tree, but I don't know it this is only to determine if you gain a skill point, or maybe if you don't drop a weapon or axe (something similar to fumble test but not in fight, but that's my speculation).
Eyesight is tested when on overland map, to notice animals and people (it is actually tested more if there are more tiles seen), maybe also when noticing tracks without explicitly involving tracking skill, but I'm not sure on this one.
Hearing is tested when on zoomed in map, for example when tracking unseen animal.
Smell probably too, though I'm not sure how important it is, I SUPPOSE you may detect the readiness of food from further distance with higher smell, but I'm not sure at all.
Also some attributes (probably Endurance, but maybe combination) are tested under certain conditions when deciding sickness or sickness level: for example flu or nausea. When I got influenza TWICE during my first summer, despite being decently clothed and warm, I'm sure it was a failed attribute test, END most probably.
Attributes also govern skill affinity (the asterisks to the right of skill), the higher affinity the easier it is to gain a skill after performing a task. I haven't noticed that affinity changes when attributes increase though, so maybe it is set only in the beginning. The "resolution" of affinity is too small to be certain.