It seems silly since there is a daub recipe that combines light gray sand and red clay, the weathered brown daub can accept phylite sand and red clay, phylite sand looks very similar to granite and wattle and daub construction is historically the kind of thing you can do from whatever random materials you have in the immediate vicinity.
It's not actually a big deal, I ended up just running about 500 blocks south and collecting some claystone sand and making some golden brown daub, just seems like a bit of a silly thing.