Coreplant

Thank you, and I did track down which mod was doing it - Dana's Tweaks. For anybody else who plays with that, you can lock doors with a padlock to keep them out after reinforcing the door using the plumb and square tool.

I can also confirm them climbing ladders, and opening doors/gates - in fact, both of those open just by the Drifters going nearby. My game is modded though, so I am uncertain if this is an Update thing or a mod thing.