Diffusion-based terrain generator

[Smog Lebottier]

So, I think there is quite a few players, including myself, that are not satisfied with the current terrain generation. It's repeating patterns giving no originality to any region you explore plus it's tendency to make big square-ish coastlines past a certain scale is immersion-breaking for me, and provide little to no incentive to explore beyond the main quest.

Now, as an old Minecraft Beta dog myself, I know full well that terrain gen is difficult, and that the current limitations have more to do with technical limits than dev choice. And it is to propose an alternate technical solution that I make this post, after seen said technical solution in action here:

 

Long story short: it's an AI diffusion-based (same technique used to generate pictures) model trained of real-world topographic data, that is small enough to run on most current gamer hardware. And the result are, imho, quite impressive.Technical principles are summed up in the video above, but for those that want to know more about it, here is the published paper of the solo researcher:

https://xandergos.github.io/terrain-diffusion/

Also, he shiped it in a minecraft mod as a demonstration, that we can find here:

https://modrinth.com/mod/terrain-diffusion

The only things it seems to lack is rivers, but given the natural erosion patterns that come out of it, maybe rivers can be placed onto it instead of trying to dig a path through it like we can see on the rivers mod?

What do you think?

[Rainbow Fresh]

I think that we don't need to bake an entire AI model solely for world generation into the game. Performance cost and the never-ending rave about ethnics and morals of gen-AI and their training data aside, plenty of people will lose their shit if the game itself is suddenly labeled as "AI". And since you brought up Minecraft as a reference, Minecraft has very nice most time beautiful terrain generations and as far as I am aware they do not run a diffusion model in the background. I'm not entirely sure what they changed since the old days to improve world generation (minus the constant slices carved into the world since the Caves and Cliffs update) other than the Nether being generated from a noise-based generator but clearly it is possible to generate all the things with nice natural variety and including rivers and oceans and huge mountains with just normal world generation tactics.

[Samu50hiderpo]

I´m agree with Rainbow, I don't think the community will view the use of AI positively.
But actually in Vintage, some mods provide pleasing results, such as Geology Addons or Terra Pretty.

[Kulze]

First off I wanna say: Yes, this is how proper AI usage looks like.
The limitations of Perlin noise (and similar ones) based generators are well known and a trouble for games which try to provide a realistic looking environment. This is a reasonable solution and isn't infringing on copyrights, which is the 'ethical and moral issues' (which don't exist to be very clear, it's solely a ignored issue since 30+ years of KNOWING those things would come in the future and never adjusting copyright and patents according to a known factor which would influence it before it happened and screwed everyone over... welcome to today hence!) mentioned often.
This is a bona fide technical implementation of using real-world satellite picturing and topographic data - which is freely available - to generate the terrain in a naturally existing way.

Introduction of rivers is also no problem there, it can be tied to elevation and tectonic shifts by tieing the weather pattern system to it, which is a major amount of work though.
As for the issue of performance... that type of generation is very performative, the majority of performance issue comes with the actual generation-side of the game rather then the decision-basis.

It also stays seed-consistent, which is important for repeating the same playthroughs. Those models are not without reason becoming widespread despite the aversion of AI-usage. They just make such a fantastic job compared to former methods that it's not even a comparison.
The performance issues stem usually from badly designed pre-generation, the Minecraft diffusion terrain generators can handle around 1700 chunks per second, which while being more GPU intensive for generation does mean the minimum specs we would see an increase how many resources are taken up, hence using it as an alternative for world-generation would be an option ensure existing players aren't screwed over with substantially increasing demands. It'll take up around 2 GB VRAM to keep it going properly, which tends to be absolutely in the reasonable range of consumer systems.
So the option to choose the trinity of weightec octave, Perlin and normalized simplex noise which VS uses as the 'default' option with a choice for diffusion-based generation if wanted would make a massive difference in world-generation quality.

But that's something for the future mostly. It would make more sense as potentially the first 'official mod' to avoid the perception issues about AI while providing the ability use the functionality nonetheless, to be merged into the core gameplay in the future.
Overall it's a good option for anyone who wants to create a mod personally to introduce it though, I don't know how far the engine allows integration, but if possible it would make a ridiculously big difference in gameplay quality if half-way decently implemented.

[Facethief]

I feel like something that’s been glossed over so far is that making any kind of world generation settings seems like it would be absolutely hell, if I have any grasp on how this would work. Also, I question how the blocks would be selected, or if the climate-based landforms we have now would be possible to implement.

And to restate what has been said by others, putting the word “AI” near the game wouldn’t be taken lightly by many, especially if you lead with “it works like AI images.” That said, there could be a way to do this without making this a PR disaster.

Let me finish by saying that Tyron has been praised as skilled when it comes to world generation. I trust his judgement on how he wants his game’s world to be.

[williams_482]

The conflation of AI-based tools like the one described (which have been extremely useful for many different research-related tasks since well before ChatGPT became a happening thing), and the "AI" LLM chatboxes/image generations which have some technical similarities but far more ethical issues and societal dangers attached, is quite unfortunate and largely a product of grossly irresponsible marketing tactics by these "AI" companies. 

Vintage Story could implement this, call it "diffusion-based terrain generation", not refer to it as "AI" anywhere, and be fine: most people who will recognize this as the product of an AI tool will understand what that does and doesn't mean well enough not to flip out about it.