A vagabond is just a human who doesn't belong to any tribe. There are vagabond villages outside of tribal territory, for instance, and vagabonds can have any of the professions that tribe members can have.
Vagabonds can be recruited as companions just the same as tribal ones, and vagabond adventurers are just the same as tribal ones.
Vagabond robbers are just as much a pest as tribal robbers.
Thus, murdering vagabonds just because they don't belong to any tribe is akin to murdering members of other tribes because they don't belong to your own (or murdering members of a particular tribe because of some reason).
The only possible difference is that I see more vagabond woodsmen, hunters, and adventurers traveling around that I do tribal ones, but on the other hand, my characters settle outside of tribal territory.
Thus, I greet vagabonds (some of them can also provide a quest, although that quest isn't exclusive to vagabonds) and theoretically trade with them (in theory, as I don't have any need to trade). Of course any "adventurer" seen on the map can turn out to be robbers of any tribe, including the vagabond "tribe", so you need to take care with those.