As of now, there's really no reason, other than it looks good, to use anything but dirt to build with. I think there should be reasons to upgrade and use better materials. Rain wearing down dirt and mud brick for example. Wind storms that can degrade thatch roofing. Stone having not only fire resistance but better insulation during winter (and summer perhaps). Drifters able to eventually break through doors and even walls etc. Make building well more than just a creative choice.