RaeIsGae

I think that the drifters are humans who were transformed from temporal instability. Them crying sometimes and raising their arms into the air hints that they're in constant suffering, which would make sense for people being crushed and transformed into horrifying creatures. I think that the rot is also still around - all perishables inevitably turn into rot - but that the rot isn't as fast-acting as the lore would lead you to believe.