Long distance water transport - Aqueduct and reservoirs

[Seyko]

Do we really need to monitor the aqueduct's condition along its entire length? It's like Schrödinger's cat. If a chunk isn't loaded (meaning there's no player there), then God knows what's going on there. It doesn't matter if it's somewhere at the source or in the middle.

This can also be attributed to the game's lore. Maybe a temporal storm is happening in a distant chunk while we're away. Or maybe Seraphim himself is setting time in motion.

When a chunk is unloaded from memory, the state is written - everything is ok. When loading into memory, the chunk is checked - everything is still ok?

[Hentai Ninja]

Regarding chunk loading and water source.

What if flowing water was given temporary status as source if source was unloaded?

Flowing water can only exist if source block is present and connected. As the chunk is unloaded the source information travels like a train to the first loaded flowing water block and is given a temporary source status to prevent flowing water blocks from dissapearing.

In other words, flowing water inherits source information and on chunk load runs an "if" statement checking if it is first flowing block and passes the source inheritance to the first in line. I dont know how laggy that would be, but if we want to go with realistic water flowing mechanics instead of really good dry/full status suggestion, it is an option.

Issue with this idea is how do we pass information from already unloaded chunk - first chunk gets unloaded, then we get the source information flowing or vice versa?

Is there any difference in information between unloading chunk's source block and player just placing dirt on top of it?

[Katherine K]

8 hours ago, Hentai Ninja said:

Flowing water can only exist if source block is present and connected. As the chunk is unloaded the source information travels like a train to the first loaded flowing water block and is given a temporary source status to prevent flowing water blocks from dissapearing.

In other words, flowing water inherits source information and on chunk load runs an "if" statement checking if it is first flowing block and passes the source inheritance to the first in line. I dont know how laggy that would be, but if we want to go with realistic water flowing mechanics instead of really good dry/full status suggestion, it is an option.

I don't think that's actually necessary. A flowing block already has to know which block is feeding it. All it has to do is check that block. If it's loaded, check if the state changed and that change needs to be propagated. If it's not, then pretend nothing changed and maintain current state.

Whether you're doing very simple Minecraft-like water where the state is just level and flow direction, or if you want something fancier, where you keep track of flow rates, this is all you really need.

[Thorfinn]

I think you are missing the point, which is not a corner case,.

Take a water system that starts at A and ends at Z. Someone shows up somewhere in the middle and deconstructs the aqueduct. That chunk is loaded, it is trivial to communicate to Z, the defined end of the system, that there is no more water. Whether it instantly stops or sets a countdown timer, at some point, the water stops. Whenever it does stop, the guy at the destination messages someone in the middle to check the first half of the aqueduct while he checks the last half. So is there water in the aqueduct when he gets there? If so, there's no point in checking any earlier. It's working fine at that point, so the break has to be after that.

The mod has to have a way to communicate to any arbitrary point in the middle whether the break is ahead of that point or behind it. I'm not sure it needs a full-blown ISAM, but it needs SOMETHING.

Edited July 30, 2025 by Thorfinn

[Echo Weaver]

19 minutes ago, Thorfinn said:

The mod has to have a way to communicate to any arbitrary point in the middle whether the break is ahead of that point or behind it. I'm not sure it needs a full-blown ISAM, but it needs SOMETHING.

Still thinking about what minimalist global data structure we'd need to track this information

It seems like the nodes we'd need are the points where the aqueduct hits the chunk boundary. Since you can split an aqueduct, this could be 1 entry point and multiple exits. For a given chunk, each entry point would need a parent exit point in the adjoining chunk. Those could contain flow/no-flow flags. Whenever an exit node for a loaded chunk is updated, the entire data structure updates, and any loaded chunks have water started or stopped.

There could probably be a delay timer based on how far away a given loaded chunk is from where the flow/no-flow updated, but that's all in the noise.

[Thorfinn]

Chunk-based blackboxing. That could work. It also presents some interesting opportunities in other things where cross-chunk data exchange might be useful. 

[Katherine K]

3 hours ago, Thorfinn said:

I think you are missing the point, which is not a corner case,.

Take a water system that starts at A and ends at Z. Someone shows up somewhere in the middle and deconstructs the aqueduct. That chunk is loaded, it is trivial to communicate to Z, the defined end of the system, that there is no more water. Whether it instantly stops or sets a countdown timer, at some point, the water stops. Whenever it does stop, the guy at the destination messages someone in the middle to check the first half of the aqueduct while he checks the last half. So is there water in the aqueduct when he gets there? If so, there's no point in checking any earlier. It's working fine at that point, so the break has to be after that.

The mod has to have a way to communicate to any arbitrary point in the middle whether the break is ahead of that point or behind it. I'm not sure it needs a full-blown ISAM, but it needs SOMETHING.

Why, though? Why does it matter? Say someone breaks a section in the middle, and not enough of the length is loaded to have that propagate to the start and end. Does it matter? When someone loads endpoints, water will still flow. What kind of a problem does it cause? If someone broke middle by accident, it's not a problem until it's a problem. If someone was breaking the middle intentionally to stop the flow, walking the length of the aqueduct to update it doesn't seem like a huge ask. If it is - you can always break the section closer to where you need the effect. As an exploit? In principle, but what's the achievement here? You'll build an entire aqueduct, just to tear it down, so that you can have magic water transport? And one that only works until enough people walk in its vicinity for the update to propagate? It's certainly nothing practical as an exploit.

In order for this to be an issue, the aqueduct will need to be hundreds of the blocks long at a minimum, with some express need to mess with the middle section without updating the ends. And I just don't see that coming up or really mattering. You just need the local section of the aqueduct to work so that you can walk its length once to make the flows connect, and past that, global correctness not being guaranteed is just not a problem.

[Crylum]

Hi everyone, added most of you comment and ideas to the inital post -> please see new sections: What material can be used to build the Aqueduct? and Problem/Open questions. Feel free to let me know what you think of the additions I made or let me know if I missed something. 

Also thanks again for putting in the effort so far!

Note that I did not yet any informations on the unloaded chunk topic you guys were dicussing, is was already 1:30am and I did not had the energy to summarize it.

 

[Thorfinn]

I'm not sure you want to propagate it, @Katherine K. There should not much difference performance-wise between going ahead and setting flowing water to 100 blocks and propagating over the next hundred blocks whenever that set of chunks is loaded. And flowing water is (or was) pretty bad. I had significant FPS drop when I used to build terraced farms, water from  a lake at the top of a hill cascading downhill with farmland on each side. Maybe 200 blocks total of flowing water.

But it really doesn't matter that much to me. I'm not going to be building a water works anyway. I'll just set up my homestead where the water is, like I do now.

Edited July 31, 2025 by Thorfinn

[Katherine K]

1 hour ago, Thorfinn said:

I'm not sure you want to propagate it, @Katherine K. There should not much difference performance-wise between going ahead and setting flowing water to 100 blocks and propagating over the next hundred blocks whenever that set of chunks is loaded. And flowing water is (or was) pretty bad. I had significant FPS drop when I used to build terraced farms, water from  a lake at the top of a hill cascading downhill with farmland on each side. Maybe 200 blocks total of flowing water.

In general, water flow gets computationally complex, because you keep adding adjacent blocks. It becomes a 3D problem. But if your water flow is mostly constrained to a path, it's a lot easier to update. To propagate water changes 100 blocks, you update a 100 blocks. If you just had water spilling everywhere, that'd be a million blocks instead.

Ideally, for high perf, you'd constraint the flow to just blocks specifically meant to transport water. So maybe some dirt channels for irrigation and stone channels to build aqueducts with. It's a bit limiting in terms of what you can build with it, but performance wouldn't be an issue.

If you want something more general, where water just flows, but can spread over very long distances, yeah, you start needing to get very clever with how and when you update things. Ideally, you want to solve for the connectivity network once, then instead of propagating the changes in water from one block to the next, you just update any sources, and all flowing blocks would be some matrix multiplied by that. You'd still have to do connectivity updates if a block changes, but it's a lot easier here to drop things you can't possibly influence. And you're still going to be working with some gnarly sparse matrices. It's a fun problem, but it doesn't sound to me like anything that complex was being suggested, and restricting water flow to blocks meant to carry water is probably enough.

[Krougal]

I think you people are over-thinking this.

Bottom line is it needs to be performant.

I don't care how cool it is to have actual water flowing through if it drops me to 5 FPS.

 

[Thorfinn]

2 hours ago, Katherine K said:

In general, water flow gets computationally complex, because you keep adding adjacent blocks. It becomes a 3D problem.

Sure but in practice, my farm was just a simple, one block wide path. And it still hammered my FPS. That's probably why flows are limited to 4 blocks, to keep things somewhat reasonable, unless you try to exploit that by dropping one y-level every time it gets to the end of its flow. Which is what my farm was. Which is why I think, even if you have a simple, one block wide path, it's sheer lunacy to try to propagate from block to block. Which is why we need a triumphant video.

I think the aqueduct cannot have virtual water in it. It just has to be rendered as if it did. It can even look as if it's flowing. Rivers mod does. But it can't be a water block. If that makes sense. But I don't care. Try it. See how it works out. I might be pleasantly surprised. I'm not going to practice my "pleasantly surprised" expression quite yet.

31 minutes ago, Krougal said:

Bottom line is it needs to be performant.

I don't care how cool it is to have actual water flowing through if it drops me to 5 FPS.

Exactly!

Edited July 31, 2025 by Thorfinn

[Katherine K]

26 minutes ago, Thorfinn said:

Sure but in practice, my farm was just a simple, one block wide path. And it still hammered my FPS.

So the water spread is not optimized to take the constraints into account. I wouldn't have expected it to be. It'd be trivial to do this optimization for irrigation channels/aqueducts in core game or a mod.

I'm not saying optimization isn't important. It's just examples provided haven't been optimized, and they absolutely can be.

[Krougal]

10 minutes ago, Katherine K said:

So the water spread is not optimized to take the constraints into account. I wouldn't have expected it to be. It'd be trivial to do this optimization for irrigation channels/aqueducts in core game or a mod.

I'm not saying optimization isn't important. It's just examples provided haven't been optimized, and they absolutely can be.

Except when you have branch upon branch upon branch in your distribution system, it can quickly get out of hand.

[Katherine K]

34 minutes ago, Krougal said:

Except when you have branch upon branch upon branch in your distribution system, it can quickly get out of hand.

Nah. Say we wanted total overkill, where we solve for the flow in the entire system. There's a continuity constraint for each branch node, and then pressure/height gradient between the nodes. Say you have, what's a ludicrous number, 10k branches? That's at worst a 30k x 30k sparse matrix to invert to solve the whole thing. (1 potential + 2 flow variables per node). First time, say on load, maybe that takes a few seconds if it's a single thread in C#. Then any time you break or place a channel, walk that change to the nearest nodes in both directions. That gives you two rows and two columns to update the inverse for. If that takes more than 10ms, there's something wrong with the code. So if you were running smooth 60FPS, maybe when you make a new connection for your waterway, you get a brief hitch with one frame jumping from 17ms to 27ms.

Then the workload per frame is to update any nodes in loaded chunks (if there was any relevant update to handle there!) and then multiply it into a sparse matrix. That's about 1ms worth of mul/add operations conservatively, so long as JIT cooperates and you don't trip on cache. So we're talking about bringing an even 60fps down to 57fps.

And this is for an absolute monster of a farm with 100 x 100 nodes of irrigation in it, meaning if you have at least 9 blocks of channels between nodes, you have covered a square kilometer in irrigated farm land? And we're considering absolutely every single node of this to be available as source or sink, which realistically you'll only have at leaves. And we're updating the network literally on a frame any change happens and updating flows on every frame as well, none of which is necessary. Split that workload between a few frames, and you won't even notice anything's happening.

 

This isn't a particularly hard problem. It's niche. Most software engineers don't have experience implementing stuff like that, but it pops up in a number of places. Physics engines deal with a very similar problem on solvers, where you might have thousands of contact points, each one acting as a set of constraint equations. A lot of computational physics deals with sparse matrices that are much, much larger without any problems as well. So the techniques are there.

[Echo Weaver]

5 hours ago, Katherine K said:

This isn't a particularly hard problem. It's niche. Most software engineers don't have experience implementing stuff like that, but it pops up in a number of places. Physics engines deal with a very similar problem on solvers, where you might have thousands of contact points, each one acting as a set of constraint equations. A lot of computational physics deals with sparse matrices that are much, much larger without any problems as well. So the techniques are there.

Vintage Story is open source, isn't it? If you have the skills to fix the flowing water performance, this is something you could contribute. It's certainly not something I have the background to tackle or other software engineers on this thread.

[Crylum]

Well, VS has a API for Modding, so my uneducated answer would be yes, you technically can overwrite functions and variables. I am not sure about open source though, if I remember correctly only a part of the source code is.
Have already used the VS model creator to create draft version for the wider block I had mentioned in my other post about irrigation canals.

Creating the required json files and mod folder structures are not very complicated, once you get the hang of it. The VS Model Creater has some minor quirks and concept you have to get used to. Setting up animations is also quite easy. But overall the tool is quite good.

My biggest problem is the programming part with C# and OOP concepts. 

Using the example documented in the VS Wiki I was actually able to implement so of the logic but it took me ages to do so, due to lack of experiences in this topic.

dnSpy is a useful tool if you want to get into VS Modding and definitely getting the mod templates form the official VS GitHub 

Edited July 31, 2025 by Crylum

[Crylum]

Existing mod: https://mods.vintagestory.at/hardcorewater

Have tested this im game, long aquedcut do not seem to be a problem

Important question: does is make any sense continuing the dicussion if a mod already exists? Let me know what you think.

Edited July 31, 2025 by Crylum

[Dark Thoughts]

I was about to post that mod too.

That being said, even without source block creation there's not too much of a point. There's not even that many natural water sources at higher elevations, let alone natural waterfalls. Disabling source block creation is also very limiting for the fact that you cannot do water based landscaping anymore. And ultimately, this is obviously all much more of an effort than just making a bucket and creating an infinite water source. Not that you really need that either, because there's annoying water holes absolutely everywhere (can't speak for the RC terrain gen).

[Seyko]

On 7/30/2025 at 11:35 AM, Hentai Ninja said:

Flowing water can only exist if source block is present and connected. As the chunk is unloaded the source information travels like a train to the first loaded flowing water block and is given a temporary source status to prevent flowing water blocks from dissapearing.

And if we go in the opposite direction, the direction of the water will flow in the opposite direction? Then the source block needs to store data about the direction.

21 hours ago, Katherine K said:

Why, though? Why does it matter? Say someone breaks a section in the middle, and not enough of the length is loaded to have that propagate to the start and end. Does it matter? When someone loads endpoints, water will still flow. What kind of a problem does it cause? If someone broke middle by accident, it's not a problem until it's a problem. If someone was breaking the middle intentionally to stop the flow, walking the length of the aqueduct to update it doesn't seem like a huge ask. If it is - you can always break the section closer to where you need the effect. As an exploit? In principle, but what's the achievement here? You'll build an entire aqueduct, just to tear it down, so that you can have magic water transport? And one that only works until enough people walk in its vicinity for the update to propagate? It's certainly nothing practical as an exploit.

In order for this to be an issue, the aqueduct will need to be hundreds of the blocks long at a minimum, with some express need to mess with the middle section without updating the ends. And I just don't see that coming up or really mattering. You just need the local section of the aqueduct to work so that you can walk its length once to make the flows connect, and past that, global correctness not being guaranteed is just not a problem.

I think so too. There is no point in complicating things.

19 hours ago, Thorfinn said:

And flowing water is (or was) pretty bad. I had significant FPS drop when I used to build terraced farms, water from  a lake at the top of a hill cascading downhill with farmland on each side. Maybe 200 blocks total of flowing water.

And what is the purpose of the water flow block in the game? Just a guess, the flow was borrowed from another block game (I see that it is not customary to call it by name on the forum) only to solve the problem with the edge of the water. As a result, the flow is not optimized and is not intended for complex things. Because there were no plans for it.

And in general, if you don't need realistic water (Welсome), then you don't need to come up with a transport system. It's enough to add "pipe" blocks that are connected into a structure. We build a line of "pipes" on a beautiful stand. At the beginning, a "funnel" block, at the end, a "spout". The end. When the state of any "pipe" block changes, the neighboring connected ones are checked. A chain reaction with loading chunks (no graphics, just math). The "funnel" is connected to the "spout" - ok. No - and there is no water.

The cherry on the cake - remote loading of chunks!

[Thorfinn]

4 hours ago, Crylum said:

Existing mod: https://mods.vintagestory.at/hardcorewater

Have tested this im game, long aquedcut do not seem to be a problem

Important question: does is make any sense continuing the dicussion if a mod already exists? Let me know what you think.

Huh. I thought that mod just disabled using a bucket to create source blocks, and was now obsolete. Obviously it morphed into something else.

[Seyko]

4 hours ago, Crylum said:

Important question: does is make any sense continuing the dicussion if a mod already exists? Let me know what you think.

While there is interest, it is worth discussing. Suddenly, while inventing the wheel, we will come up with a spaceship. Or vice versa.

[Thorfinn]

5 hours ago, Crylum said:

mportant question: does is make any sense continuing the dicussion if a mod already exists?

Don't know about y'all, but once I find something I usually stop looking for it.

[Krougal]

5 hours ago, Crylum said:

Existing mod: https://mods.vintagestory.at/hardcorewater

Have tested this im game, long aquedcut do not seem to be a problem

Important question: does is make any sense continuing the dicussion if a mod already exists? Let me know what you think.

LOL, how did we all miss this mod?

[Facethief]

5 minutes ago, Thorfinn said:

Don't know about y'all, but once I find something I usually stop looking for it.

It's so funny how things are always in the last place you look!