I haven't beat the game yet, but from snippets I'm getting from books, the behavior and appearance of the Drifters, the ignorance of the traders, the Tinkerer class, and finally the Eldritch-being seems to point to the fact they are not what they seem.
My theory is that the Drifters are creatures similar to those found in the movie '9', which is about tiny burlap dolls possessed by pieces of the inventor's soul to fight against a rouge military AI. But in this case, a clockwork AI was created to assist with stopping a zombie plague. In the end, a search for a cure was abandoned, and instead the inventor and the AI decided on an alternative to save the world: TIME TRAVEL.
However, while the zombie outbreak was stopped via an incursion by Drifters created by the clockwork AI, the AI was driven mad by the suffering the Drifters went through due to paradoxes tearing them apart, multiplying them. The AI realized that, due to the paradox, it had to make a choice: Either it succumbs to being forgotten, and humanity is saved, or seize the future for itself. Considering the pain it was going through (which would normally be impossible for it to feel), it did not feel particularly heroic. Its creator attempted to stop it, but all they could manage was preventing it from escaping. But they themselves were overwhelmed by the Drifters. So, the Drifters replaced the zombies as the threat, and most people assumed the Drifters were related to the zombie plague, because of course they never knew about an AI that was never invented.
Now that time is broken, and the world is slowly falling apart...eventually, the AI will be claimed by the temporal apocalypse itself, and be turned into the eldritch being we see during temporal storms. However, while the end of the world is one possibility, one side effect of time falling apart is people who would've normally been around during the zombie apocalypse have been yanked from that original timeline, to the one where the AI has destroyed civilization.
Now both timelines compete to exist.
This is both a lore theory, and perhaps a suggestion to the developer as to where to take the story. I kinda want multiple endings like Undertale, with the possibility to make peace with the AI (after many attempts, Naruto style). Alas, no matter how hard you try, only one timeline can exist...and the clockwork AI knows this.
Last thing, my friend would utterly be devastated by an entity that steals peat. He makes comically high towers outta peat. They annoy me, so next time he does it I'm going to take and hide all the peat and say the 'Hollow Watcher' did it.