Feature request: allow prioritizing collider generation over visible meshes #87
Comments
|
The terrain has collision as soon as its meshed. Collision is built along the current LOD mesh surface, not along what the mesh will eventually look like. It's likely that you're falling through the terrain when you are standing on one LOD and it switches to another with a different enough shape that you're on the other side of the isosurface. Thus you're on the other side of the collision shape as well. I think prioritizing building LOD 0 around the character is a better request. I think some of that is done, but maybe not enough. |
Yeah, something like that. Is it not possible to generate the collider before the mesh, or does the mesh have no additional data? |
|
The engine generates the collision mesh from the vertices of the display mesh. They are essentially equivalent. I believe you are asking for the fine detailed LOD 0 collison mesh generated independent of the higher LODs display mesh (lower detail). So it will calculate the mesh data twice instead of reuse what has already been calculated. That also means the collision is a different shape than what is shown in screen. If the mesh data is known so it could have a collision mesh generated, then it could also be displayed. Finally neither the engine nor Zylann's module is designed that way and it doesn't sound more efficient than what is currently done. |
|
Right, makes sense to me. Then, is it possible to force collision to be at a lower level near the character, e.g. LOD 1, so only the mesh exists in LOD 0? That would make generating terrain slightly faster and LOD 1->0 conversion, which would be pretty common, wouldn't cause gameplay issues. Side note: how much more RAM expensive is it to have both a mesh and a collider rather than just a mesh? |
Marginal. You can look yourself by using the monitor inside Godot and enabling it disabling collision. Only LOD0 should exist around the character at all times. The collision LOD should be identical to the mesh at all times. If you can run quickly outside of the LOD0 sphere and have other terrain sections be updated before the one your character is on, that is a scheduling issue that could probably be worked on. Perhaps it could track the velocity of the viewer path and prebuild in that direction. I don't see how forcing LOD1 around the character or forcing collision and the display mesh to be different LODs is more efficient or provides a better player experience. Both require dual generation of the section in LOD1 and 0 when only LOD0 should be generated, once. |
Perhaps when LOD 1 collision is generated, it stores it in a way that makes it easy to transfer into LOD 0 with the quality of LOD 1. I'm not exactly sure about the specifics, but I'm aware of the blocks having a different size. |
|
More control over which block LODs produce collision meshes is expected. However prioritization isn't really a thing right now, because there is nothing to prioritize them over from (i.e visual mesh and collision mesh are the same mesh, and if you can see a high-res mesh, the low-res mesh is always available already). |
|
Everytime I load up my noise_smooth_lod demo I see several LODs load around the player, before LOD0. Currently it takes up to 5s for several LODs to be built before LOD0 is finally finished. That looks like room for improvement. We only need LOD0 around the player. What it looks like is the entire landscape is built at LOD5 (say). Then a smaller sphere is rebuilt at LOD4, then a smaller at LOD3, and so on until LOD0 is finally built around the player. IMO it should be reversed, and the extra, not displayed LODs not generated: LOD0 sphere built out first. Then outside that sphere is LOD1 only, then LOD2.... If the player moves, new LOD0 blocks are built and old LOD0 blocks are replaced with LOD1, and other rippling effects. No building multiple LODs for the same location. No building other LODs or locations until LOD0 is built around the player. Also you could:
|
|
In the first implementation I attemtped to load all lods preemptively in concentric areas, much like The second attempt is the octree we currently have, which works in a very similar way to what Google maps does (not really a coincidence :p). Following the constraints of octrees, it cannot subdivide if the future children are not ready, and cannot unsubdivide if the future parent is not ready (if it did, you'd get a hole). This provides the property that if a block is loaded, then blocks of lower LOD will be guaranteed to be loaded. This property is used in editing, because we know if a LOD0 block changes, it can ripple down to all lower lods without having to defer anything. The downside is what you described: you can't tell the loader to load LOD0 in advance if LOD4 is not even loaded in the first place, because that's how octree traversal works. The look-ahead is only one level. The initial phase then looks slower than it could be. It could indeed be optimized by traversing it unrestricted, and then reversing lods if you don't move (like a preload phase). But the case where you move as blocks are loading must be considered. I may take an improvement pass at this system at some point, to see how it can be done in practice, and what the side effects are. Note: this issue is then about two different topics so far:
|
|
Back to this, I guess gmail stopped providing me notifications. Thanks, gmail! The discussion about how higher LOD levels have to be loaded before LOD0 is interesting. What would happen if you queued the mesh generation and just traversed to LOD0 quickly, then put LOD0 generation in front? Edit 1: I already do the freezing the player thing. It's just pretty annoying to get it in gameplay. Edit 2: Yeah, I'm editing this a lot. I'm trying to get back to this conversation after the time that passed. Anyway, having LOD1 for collision and LOD0 for the visible mesh would probably help a lot. It would mean that we have to worry less about weaker devices. And besides, LOD1 would be easier to calculate collision with to begin with. |
I'm not sure to understand.
This should be a last resort situation. If you get it often you need to improve the other things (generator speed / player speed limitation). Computer has limited resources. Even if LOD0 were prioritized (or if there was no other lods) it won't get rid of this. You could even be playing this over network, so it has to be expected. Also note that if LOD priority is reversed making LOD0 be calculated first, but only LOD1 is used for collision, it means collision will actually arrive later than LOD0 (quite a mess to handle if you still want it before somehow). Choosing LOD1 over LOD0 for collision would only bring benefit with the current octree. |
|
I started prototyping an alternate octree implementation, along with a fake reproduction of the terrain loading behavior. For example, if you wait long enough for the world to load around you, then move fast, we could think the loader should still attempt to prioritize LOD0 as fast as it can around you. The result is, when looking at LOD grids, you would see a "trail" of LOD0 blocks following you, in the middle of a background composed mostly of LOD3/4 blocks.
It means those LOD0 blocks cannot be made visible straight away, because it would break the neighboring rule: seams between LOD0 and LOD3 are not supported. The lower LODs around must split first into higher LODs, until they can safely border LOD0... which brings us back to the original design: you need to load LOD3, then LOD2, then LOD1 before you can show LOD0, not the other way around. Physics doesn't support any seams so it could turn on straight away, but there isn't much to do about visuals. Worse, they would "feel" delayed due to LOD0 being prioritized for something invisible / yet to become visible (if it ever does). If the goal was only to have a small island of early LOD0 blocks around the player just for physics, it would not even need to change the current octree and be done separately (only viable if only this LOD has physics). I haven't fully explored the possibilities, but so far it's not really working out. |
|
:/ Well hopefully FastNoiseSIMD and Godot 4 visual server rewrite will improve things. |
Interesting. Now, as for visuals, I wouldn't say they matter that much since the player is travelling so fast. Is it possible to then do that forced LOD3 - 0 generation but in reverse order? So, in terms of generating the chunks, only a single chunk of loading LOD0-3 would be prioritized. To make it clearer, 0-1-2-3, 0-1-2-3, 0-1-2-3. This prevents the player from getting really deep, reaching even higher LOD levels. Perhaps something could be done so that you can keep that speed longer before you eventually catch up, while still preventing getting really deep... I might have to experiment with a similar system later.
Adding LOD1 collision to the mix, perhaps since LOD1 is used as collision, it technically doesn't matter if the player is on LOD0 or LOD1. So we'd only need to force loading LOD1, which is then followed by LOD0 purely for visuals. This would increase the amount of speed you would need to break the system. At that point, since the player is so fast, it probably wouldn't be much of a surprise for the game to have to pause in order to generate the terrain - getting there so quickly would probably be worth a whole loading screen. So, combining these two ideas: instead of generating LOD3-2-1-0, generate it in order 1-2-3-[0]. 1-2-3 is for being able to go deep, which then is followed by 0's visuals which don't affect gameplay, but only if the player didn't attempt to move out of LOD1 again. |
Again, this assumes you start from nothing. If not, it would slow down visuals even more. You would not see anything anymore if a larger LOD has been generated beforehand in such area, since visuals work by splitting to respect the neighboring rule. Letting the octree split to its target shape so we know in advance what to actually load sounds interesting, and doing that in initialization phase would be a good way to speed up spawning, but I could not find a proper approach for the rest yet. So far the only thing that sounds feasible is to have two sources of block schedulings: one driven by the octree as usual, and another driven by an area around the player at LOD0 or 1, for which the purpose is to provide early collision. And yet, if you still move fast enough to go past LOD0, it would have very little use and slow down the lower lods when flying far above the surface for example. Things might sound simple but changing the way the current octree works ripples through a lot of edge cases and emergent behaviors. IMO the real bottlenecks currently are discarding colliders for lods above 0, 3D noise generators (GDScript ones being the worst), and the upload of meshes which are embarrassingly done on the main thread. |
|
I'm pretty much brainstorming ideas here. Actual implementation is different, of course. So I'm going to assume that it attempts to connect the LODx chunk with the LODx+1 chunk by modifying the vertices at the edge. In that case, could a system be designed that it doesn't only handle LODx+1, but LODx+y? To prevent it from looking too ugly, the vertices would slowly pass from x to x+y, let's say one row at a time. If this is done, a better implementation of prioritizing LOD0 while making it look as good as possible might work. Once again, I'm bringing up how it wouldn't be 0-0-0-0-1-1-1-1-2-2-2-2-3-3-3-3 in the scheduler, but rather 0-1-2-3-0-1-2-3-0-1-2-3-0-1-2-3. I marked the latest added "chunk" in bold. So it would generate LOD 0 first, going through and eventually finishing LOD 3, then working on the next forced LOD 0 chunk afterwards. If we can connect theoretically any low LOD with any high one, if the player is moving fast enough, it could be modified to become, for example, 0-1-3-0-1-3-0-1-3-0-1-3 instead, to skip generating an unnecessary LOD level, and only generate LOD 2 when it seems alright. Alright, but what happens to the remaining LOD levels? Well, they also get added to the back, with only a slightly lower priority than the one that's generating collision and a "neat enough map". Let's say it becomes 1-3-6-1-3-6-2-4-5-2-4-5-0-0. Yeah, something like that should work. Just for fun, let's describe this situation. Now all the 1-3-6s necessary for the "collision and a neat enough map" are complete. The game is generating the rest of the LOD levels. But oh no, we cut it in the middle. The queue is currently 5-2-4-5-0-0. Since it currently isn't generating a necessary LOD, it proceeds to add the levels like this: 1-3-6-5-2-4-5-2-4-5-0-0-0, and the game hopefully doesn't have to freeze until it gets generated. Well, that ended up being REALLY long. But you get the idea. |
Zylann
added
enhancement
Needs research
That surely lifts the constraint, but I got new ideas since, still without really changing octree evaluation order: Intermediary blocks could be skippedStarting from a given octree shape, if we want to move somewhere else, we may have to transition it to a different ideal shape. Reaching such a shape requires to join or split one or more levels of nodes, with the usual split policy ensuring the bordering rule (i.e "distance"). This is currently done step by step, but some intermediary steps are not really required. Indeed, if we look at the visible LOD maps after loading is complete, we can see concentric rings of blocks converging to the viewer. All blocks located in the ring's "holes" are not needed in the final result, because that space is occuppied by the next LOD. Yet, they are loaded anyways, and they will eventually be used when we start moving. Most obvious one is LODmax. We will never see LODmax if we are within the region of the octree, so it's pointess to calculate it. To ignore them, we may first evaluate the octree to split every node we need to split, without affecting blocks yet. Now we know what to load down to LOD0, we do a second pass in which we will trigger the loading of all nodes that aren't split. This way, only the absolutely required ones will be scheduled for loading, skipping the others. They will still be sorted based on distance and LOD. I don't know if this solution is good yet, because it has two caveats:
Joining a LOD does not need to invoke the generator againIf we have 8 blocks of LODn that need to be joined into a lower LOD, we can simply produce LODn+1 by downscaling the 8 blocks using nearest-neighbor samplingm, using the data we already have. It works because that's what the generator would do anyways, and is even assumed by Transvoxel. Those ideas remain centered on the LOD octree system, but I still believe streaming speed isn't only related to it. It's a compromise with pros and cons, designed to work relatively well in both static and dynamic scenarios. As I said earlier, there are other obvious sources of lag which need to be addressed (and are easier to fix). I'm writing it down here for later reference, because I'm slowing down my activity. I've been very active for weeks mostly due to Transvoxel, and this is my free time which I'd like to spend on other things :p |
Indeed, it has to. I guess it's easier to just deactivate 8 corners than have lots of tiny ones, right? Or, actually, could a shader be used to simply disable parts of the world, rather than deactivating them through code? Though this would likely only work for LOD levels that are higher than the max collision LOD level.
So I'm going to assume this is to do with how you see the high LOD before the low one loads below you. Indeed, it's pointless to calculate this. In this case, it is an advantage to reverse the LOD loading order, since the terrain under the player will appear first, then the rings around it afterwards.
Could it somehow be calculated using the vertices then? But making the map threadsafe might be the option to go for here instead.
Of course. Every bit helps, though. |
|
I just realized there is already a setting to turn off collision generation beyond a given LOD, which speeds up meshing. It's called I added a note in #107 regarding the neighboring rule. It might be doable to produce higher LOD first regardless as an option, at the expense of producing z-fighting/overlapping meshes. |
Yeah, I have this active. The point is to also add a minimum bound, so LOD0 collision won't get generated. Anyway, had this thought a while ago. Thinking of neighboring, what if we had something like this: try to generate LOD0 for a given area, which then checks the neighboring chunks and if they are a LOD level higher than 1, force generate those first? If we're using chunks the same size as LOD0, this would only force LOD1 to generate, as that LOD1 block is fully surrounded by LOD2. To demonstrate what I mean: (X is the currently force generated chunk) After the force LODs are dealt with, the typical generation can continue. (I love editing just because I don't put 2 newlines after a quote...) |
|
I think it is a mess trying to solve a LOD problem in realtime. There are not enough computational power to generate a lot of LODs layers. |
It's complicated but it's manageable. The current system has still ways to improve. Other voxel engines do it just fine with a similar approach (UE4 Voxel Plugin)
But this is not even possible. And if it was, you'd start killing performance of the mesher, which would have to first generate a high-density mesh up to kilometers ahead, only to be decimated later, which is a waste. Right now it doesn't need to generate high-density immediately, and it doesn't need to decimate either. |
I've been messing with the terrain for a while, speeding through it with a little controllable capsule. It seems that there is an issue where if the capsule is on terrain that is modified, there is a chance that it might end up inside of it.
So I have a mechanism set in place that would pause physics until the high-resolution collision is generated (together with collision LOD level set). The feature I'm requesting here is basically where the system would prioritize generating these colliders for example where this capsule is, so it takes less time for the game to unpause, or maybe I could even set up something to make sure it basically never happens. For this, I don't even need the visible mesh to exist yet, the goal is to make sure collision at a certain chunk is a thing. (That last statement could be toggleable, so visible meshes could be forced together with colliders as well?)
Anyway, this should be applied to the viewer object and potentially wherever else the script wants to force collision to be generated more quickly.
The text was updated successfully, but these errors were encountered: