Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Occlusion Culling / Fast occlusion for outdoors #22048

Open
DevMagicLord opened this issue Sep 13, 2018 · 24 comments

Comments

@DevMagicLord
Copy link

commented Sep 13, 2018

An occlusion system, multi threaded able to do fast occlusion on outdoor levels.
When there is many houses, characters an dense forest, perhaps inspired by some of those principles ?
https://www.guerrilla-games.com/read/decima-engine-visibility-in-horizon-zero-dawn

An optional baked occlusion would be great when objects tagged as static will never move.
This would save gpu resource.
Something inspired by Umbra ?
https://www.gdcvault.com/play/1020231/Solving-Visibility-and-Streaming-in

@pgruenbacher

This comment has been minimized.

Copy link

commented Sep 14, 2018

huh witcher 3 only has 16 people on their engine team. That's impressive. I guess that's the way it is with most AAA engines though is you just build teams of senior engineers.

reduz I know has mentioned wanting to implement the occlusion logic. However I think that this only matters with high #s of static mesh objects, and if you're doing high # of static mesh objects there's a bunch of other optimizations to be done in tandem. Batching identical objects as an example. You also don't want hundreds of scene tree objects for your static objects, since that's just wasted cpu time navigating the tree. I'd imagine you'd instead have a StaticEditor and StaticRenderer nodes. The StaticEditor allows for building a level(s) of static geoemtries using custom editor plugins, and the staticRendere node accepts the data and renders it with as much optimization as it can. The exported game would then just pass the level data to the staticRenderer. That way you'd have as much optimization possible for that sort of rendering using custom nodes without having to mantain the flexibility and compatibility of the game engine.

@pgruenbacher

This comment has been minimized.

Copy link

commented Sep 14, 2018

I guess my point is godot would be better off by making it possible for third-party plugins to do this sort of optimization rather than making it an overarching feature. E.g. you could design your 3d level, and then parent it under a "StaticRendererPlugin" node which would parse the scene tree to find all the mesh instances and mesh resources, bake them all into an optimized dataset, and then render the meshes using occlusion and multimesh batching and resource-streaming. It would also turn off any sort of scripting/signals attached to the mesh instances since those would not be compatible with using a single-node to render anything.

@pgruenbacher

This comment has been minimized.

@pgruenbacher

This comment has been minimized.

@pgruenbacher

This comment has been minimized.

@reduz

This comment has been minimized.

Copy link
Member

commented Sep 14, 2018

Here are my views about it. I did a lot research on occlusion culling over the years and implemented and used many techniques. Most common options (not necesarily exclusive) are:

  1. Portals and Rooms - Great for interiors, requieres small amount of manual work. Pseudo real-time (you can move rooms and portals around)
  2. GPU depth test / conditional rendering - not too efficient because it uses conditional rendering, has a frame delay and is really complicated to implement. fully real-time.
  3. Reprojected depth buffer testing on CPU - Works very well for exteriors, not as well for interiors (but not erribly). Fully real-time, but does not support dynamic occluders well.
  4. PVS, which requires preprocessing, out of which Umbra is the most popular. Only static geometry.

I have used Umbra for large projects in Unity, and honestly it's an enormous pain in the ass for interiors, even though it's advertised for it. Small mistakes in materials can lead to holes, which break the entire culling.. and you can't really tell until performance dies. For exteriors it works better, but you can't really have super big worlds made with it easily. I think games like Witcher only used it for towns and cities. In general, I don't really like PVS solutions much. It has the disadvantage that any change to the world requires recomputing everything.

This is on the roadmap for the next Godot release, and my plan is to implement 1) (portals/rooms) and 3) Reprojected depth buffer (and it works superbly well with octrees, which Godot uses). I think this combination will be enough for far most games. Not going to use anything pre existing because this is something that does not requiere much code and needs to work very tightly integrated to the engine, while using an external library would mean duplicated classes and a lot of glue for no benefit.

@pgruenbacher

This comment has been minimized.

Copy link

commented Sep 15, 2018

Hydock's email reply. I didn't actually realize the custom render pipline was available. So happy.

This kind of thing is why Unity added its programmable render pipeline. Godot has the same kind of functionality through the ability to make custom render servers.

http://docs.godotengine.org/en/3.0/development/cpp/custom_godot_servers.html

If you have specific needs or tweaks on how your scenes should be rendered for optimal performance that's different than general use-cases, you can program something to your games needs. If you find it's a good enough thing to become reusable, such as optimizations specifically for large open world games, you can always release it on the asset store or sell it.

@pgruenbacher

This comment has been minimized.

Copy link

commented Sep 15, 2018

@reduz

This comment has been minimized.

Copy link
Member

commented Sep 15, 2018

ok guys, keep discussing, I won't interrupt.

@nhydock

This comment has been minimized.

Copy link

commented Sep 15, 2018

sorry, I deleted the comment because the more I read into it the more I felt like I didn't actually understand what was going on/present. @reduz knows more about it of course because it's an engine design thing, and the docs there were about engine development. I was wrongly under the impression that it was something that could plug-in like unity's scriptable render pipeline.

If godot could expose overriding low-level things like the visual server through plugins instead of modules that require recompiling the engine, then I feel this discussion could be easily addressed as we can leave it to the community to make game style specific tweaks for rendering.

@reduz

This comment has been minimized.

Copy link
Member

commented Sep 15, 2018

There really is not a sensible way to expose low level things without limiting the architecture or performance, imo, an these are things that should be provided by the engine itself, not extensions.

@ghost

This comment has been minimized.

Copy link

commented Oct 13, 2018

This is on the roadmap for the next Godot release, and my plan is to implement 1) (portals/rooms) and 3) Reprojected depth buffer (and it works superbly well with octrees, which Godot uses). I think this combination will be enough for far most games. Not going to use anything pre existing because this is something that does not requiere much code and needs to work very tightly integrated to the engine, while using an external library would mean duplicated classes and a lot of glue for no benefit.

@reduz , Wait a minute ... what do you mean "This is on the roadmap for the next Godot release"
do you mean this will be in 3.0.7 ? or 3.1 Beta ? or 3.2 ?

@akien-mga

This comment has been minimized.

Copy link
Member

commented Oct 16, 2018

@CCODE05B 3.2.

@Chaosus Chaosus changed the title Fast occlusion for outdoors Occlusion Culling / Fast occlusion for outdoors Apr 8, 2019

@Chaosus

This comment has been minimized.

Copy link
Member

commented Apr 8, 2019

Added "Occlusion Culling" to name so it can be searchable for other users to avoid duplicated issues

@HeadClot

This comment has been minimized.

Copy link

commented Jun 24, 2019

This is on the roadmap for the next Godot release, and my plan is to implement 1) (portals/rooms) and 3) Reprojected depth buffer.

Hey @reduz is this still on the roadmap for 3.2? Asking as I know you are busy working on Vulkan.

@clayjohn

This comment has been minimized.

Copy link
Contributor

commented Jun 24, 2019

@HeadClot occlusion culling will definitely not be in the 3.2 release. It will be in 4.0 or later.

The reason for this is that 3.2 will still be using GLES 3. And we dont want to implement it for 3.2 and then have to completely rewrite it for 4.0.

@DevMagicLord

This comment has been minimized.

Copy link
Author

commented Jun 25, 2019

Alternative occlusion options like portals or scene depth buffer are also welcome, and could be enough depending on the game type and size.

I don't understand why there are is so much work on graphics, instead of working on optimizing the base to get lot better performance with good occlusion, and navmesh pathfinding multi threaded with tools as other engines standards.

@clayjohn

This comment has been minimized.

Copy link
Contributor

commented Jun 25, 2019

@DevMagicLord have a little patience. Optimizations are coming. It doesnt make sense for someone to volunteer huge amounts of time to implement occlusion culling for 3.2 only to have to rewrite it for 4.0. So the current plan is to wait until Vulkan is ready and then do occlusion culling and everything else after.

@DevMagicLord

This comment has been minimized.

Copy link
Author

commented Jun 25, 2019

@clayjohn I learned to stop asking and waiting for features, and stay focused on software that suits my needs, this means other 2D or 3D engines that suits them. So i'm really not in any hurry or lack any patience LOL

My point of view about Vulkan is it won't change dramatically the graphics.
while stylized non realistic indie games would benefit lot more from occlusion culling for performance or a complete navmesh system and tools for solid pathfinding.

About @HeadClot proposal to bring portals and depth buffer, i also think it's a good idea and welcome to bring some early and alternative solutions for people to test, that could be suitable and enough for some games.

But i really don't care if that's not worked on before Vulkan is completed, and i think it's lot better to take all time needed to make culling and navmesh the best possible instead of rushing those features.

@HeadClot

This comment has been minimized.

Copy link

commented Jun 25, 2019

@clayjohn We already had rooms and portals in Godot 2.1. The code is written. All that some one needs to do is to port it to Godot 3.2 which is all I am asking. This should be fairly straight forward.

#29160

@clayjohn

This comment has been minimized.

Copy link
Contributor

commented Jun 25, 2019

This should be fairly straight forward.

Implementing new features is never straightforward. The underlying renderer for 3.0+ is completely different from 2.1 and the one for 4.0 is going to be substantially different as well. Also keep in mind, who are you speaking to here? Who is it fairly straightforward for? Surely not a new contributor, otherwise you would do it yourself. The very fact that you haven't done it yourself means it is more work than you are willing to do. And if you are not willing to do it, why should someone else do it for you?

If someone were to implement rooms and portals for 3.2, it would have to be completely rewritten for 4.0. So why should someone volunteer their time to work on a feature for you that will have to be completely rewritten in a few months?

Sorry to be so negative, but it is worth being clear about this. On a positive note, if a user does come around implements these rooms and portals or occlusion culling in a robust way then it may be able to make it in time for 3.2. So if you absolutely need these features, then you can do your best to find someone willing to add them, pay someone to add them, or add them yourself!

@DevMagicLord

This comment has been minimized.

Copy link
Author

commented Jun 25, 2019

I agree, it's better to wait for Vulkan, or this would be wasted work because the culling system should be completely re worked for the new renderer. Better take the time to make it good and not rush.

@kakoeimon

This comment has been minimized.

Copy link

commented Jul 2, 2019

Does this means that the same optimizations are going to be used for gles2 and vulkan?
I am not a graphics programmer but it's obvious that this is not the case.
The problem is that when Godot 3.0 came out we were promised that we are going to get Occlusion culling in 3.1 along with gles2. This made sense back then.
When 3.1 came out Occlusion culling (portal and rooms) had been pushed to 3.2.
Now 3.2 is around the corner and occlusion culling is pushed to 4.0
I bet that Occlusion Culling will be pushed to 4.1
But in the end it's going to be that the promises that it's going to be back in a few months turned out to years.
It turns out that this is a work that none is willing to do and it is just pushed further and further, and for me this turned out the biggest dishonesty in Godot.

@lawnjelly

This comment has been minimized.

Copy link
Contributor

commented Jul 2, 2019

This should be fairly straight forward.

Implementing new features is never straightforward. The underlying renderer for 3.0+ is completely different from 2.1 and the one for 4.0 is going to be substantially different as well.

Yes I very much doubt it would be anything like 'porting' from the existing code. That said I am aiming to have a little look at this (or at least the feasibility) when I'm done with the timing system. It is possible I might make a simple room / portal / pvs system as a module, but it is likely to be replaced once reduz etc get to do it properly, as it might ideally need integration with the editor to be more user friendly.

The biggest obvious hurdles are loading in the portal / room data, and finding whether there is a way of quickly turning off visibility and physics for branches of the scene graph. There is a visibility flag but no simple way of turning off physics that I've encountered so far.

That said rooms / portals is likely to be a bigger win in indoor environments. Outdoors is generally a nightmare, some other occlusion technique might work better.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
10 participants
You can’t perform that action at this time.