My technique, at least early game, is to watch the current temporal level and go caving on a day when it says calm- it feels like a huge jump in how many drifters you encounter underground even just between low and calm. I don't know how much this holds true once you start going super deep- I'm still too scared for that lmao. But it works great for clearing out near-surface caves when drifters are still two-shotting you.