I do not think the drifters were human at any point.
They don't have faces, just big gaping toothy maws. Their interiors are all metal as are their digits. Their exterior is made of flax, implying it was a manufactured automaton rather than starting as a sentient being.
So, I think they might've been golems, invented by Jonas when he was doing his mad scramble to save his people from the Rotpocalypse. And of course he notices early when those golems turned violent with the implication that they'd started killing some people.