Implement tessellation as an alternative to parallax occlusion mapping

[Arnklit]

Describe the project you are working on

Various 3D projects, including the Waterways add-on

Describe the problem or limitation you are having in your project

It's difficult or impossible to manage high density details in Godot at the moment. Especially with non-static meshes such as water, hair, deforming snow.

While POM can help with adding fine detail it has a lot of limitations:

• POM doesn't change profile of the mesh, causing the effect to fall apart at certain angles.
• POM doesn't cast shadows from the added displacement (this might be solved).
• POM displacement doesn't intersect correctly, causing the effect to fall apart when objects are places on top of or intersecting with the POM surface.
• POM doesn't work with tri-planar texture mapping, limiting it's use for environments (this might be solved).

While Godot 4 can handle a lot more geometry than Godot 3.x, I don't think it's feasible to add all this detail into the meshes for most users. Even if Godot eventually gets some kind of virtualised geometry solution similar to "Nanite", it is often much easier to add details with tiling textures than using dense meshes, both from an asset creation point of view and dealing with the asset import / storage.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

An option to enable dynamic tessellation on a material or mesh.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

I am aware that geometry shaders are getting deprecated, so this would have to be a custom compute shader implementation, but I do not know the details on how this would be done.

If this enhancement will not be used often, can it be worked around with a few lines of script?

This would be used often and cannot be done easily with a few lines of script.

Is there a reason why this should be core and not an add-on in the asset library?

This has to be done in core.

[Arnklit]

Note I am by no means a graphics programmer, but researching this a bit more, it seems it is possible to implement into the core of Godot 4.0's Vulkan renderer without geometry shaders by implementing tessellation stages.

https://registry.khronos.org/vulkan/specs/1.3-khr-extensions/html/chap22.html

[Calinou]

In a way similar to geometry shaders, tesselation has lost a lot of traction over the years. These days, it's more common to use actual geometry to represent bumps in surfaces where it matters (plus a LOD system or virtualized geometry). Modern GPUs can handle a lot of triangles as long as they're not smaller than a pixel (which causes those triangles to be rasterized through a slow path). On the other hand, a lot of GPUs¹ implement tesselation in a slow manner, making the added implementation complexity not worth it in most cases.

• POM doesn't cast shadows from the added displacement (this might be solved).

See #1942. I've tested godotengine/godot#65307 and it doesn't resolve the issue with POM shadowing, but that could be done with further changes.

• POM doesn't work with tri-planar texture mapping, limiting it's use for environments (this might be solved).

This can be implemented in a shader, but it's very expensive due to the high number of texture fetches required. To my knowledge, using tesselation won't resolve this issue, as you still need to perform those texture reads to modify the underlying geometry.

Footnotes

1. Anything that's not NVIDIA 🙂 – This is one of the reasons why many AAA titles in the early 2010s used excessive amounts of tesselation. ↩

[Arnklit]

@Calinou do you have any details on the performance thing with NVidia? Wouldn't that be when using geometry shaders?

There are modern examples of tessellation made for consoles only, which are AMD chips. Like the Demon's Souls remaster.
https://youtu.be/kAv-ajC1bWg?t=365

[ja2142]

Tessellation being slow on non-nvidia GPUs is a valid concern, but I think there are cases in which this is the most sane approach: one of them is having a large geometry with fine details in which both close-up and far away elements may be visible at the same time (prime examples of this would be terrain, large bodies of water etc.). In those cases other methods I know either don't work or are rather hacky:

• LOD works per mesh, that's not helpful,
• dividing large mesh into chunks to take advantage of LOD wouldn't be that bad, except it generates more problems on chunk boundaries.
• POM is kind of ok for adding really small details, but doesn't do anything to geometry, so say a low poly mountain occluding something farther away would look terrible.
• There's a moving clipmap idea (https://www.youtube.com/watch?v=rcsIMlet7Fw) which, while very clever, requires moving a terrain, even if it is stationary, which spills rendering complexity outside of the shader itself.
• Virtual geometry/nanite type of solution seems really nice, and it would solve most of the terrain problem. Main issue with this approach is that it isn't implemented in godot at the moment. Even if it was though, I'm not entirely sure how good it can work with animated or (partially) procedurally generated models.

So unless tessellation performance makes it unusable, I think it'd be better then all of the solutions above in cases like terrain and water.

@myaaaaaaaaa you propose to run tessellate() before vertex(). In both vulkan and opengl tessallation seems to be a step after vertex shader. So the pipeline probably has to be done more like in those apis:

vertex -> tessellation control -> tessallation evaluation -> fragment

Tessellation evaluation coule be named something like interpolate()?

There's a quite nice sample of tessellation shaders in https://github.com/KhronosGroup/Vulkan-Samples in case someone tries to implement this:
https://github.com/KhronosGroup/Vulkan-Samples/blob/master/samples/api/terrain_tessellation/terrain_tessellation.cpp
https://github.com/KhronosGroup/Vulkan-Samples/blob/master/shaders/terrain_tessellation/terrain.tesc
https://github.com/KhronosGroup/Vulkan-Samples/blob/master/shaders/terrain_tessellation/terrain.tese

[PeterSHollander]

I would like to add support for seeing tessellation implemented in Godot 🌐

In my use case, it would be a major fidelity boost for curved model silhouettes in XR content (see PN-Triangles for reference), where it is not predictable how close the camera will ever be to any object, and where tessellation would save an otherwise unreasonable amount of LOD file size to achieve a similar result. Additionally, as suggested here, using realtime tessellation for a large amount of models would save significantly on memory usage. A use case of tessellation shaders for skewed viewing angles of large meshes such as terrain can be found in #6279 (comment) and #445 (comment). Silhouette tessellation may also relate to the implementation of OpenSubdiv proposed in #784, but as it operates realtime on the CPU it is my understanding that GPU realtime tessellation would be more performant (and potentially more flexible for non-uniform tessellation).

I would also like to ask for clarification if there is a possibility to achieve tessellation-like behaviour using compute shaders, as suggested by @fire in #2075 (comment), and adjacently suggested in this comment?

It appears that @painfulexistence is experimenting with adding tessellation, as mentioned in #6121 (comment). While I am not graphics engine programmer, I am willing to learn if it can help contribute the feature of tessellation to core Godot.

[fire]

https://github.com/v-sekai/godot-subdiv needs a implementation of vulkan compute shaders.

See also PixarAnimationStudios/OpenSubdiv#1153

Friendly note. The currently implementation of opensubdiv requires a gdscript plugin. I wasn't able to find effort to port it to C++. It should be in c++ though.

Edit:

Ported to godot engine c++ modules.

https://github.com/V-Sekai/godot/tree/vsk-subdiv-4-0

[erickweil]

I am aware that geometry shaders are getting deprecated, so this would have to be a custom compute shader implementation, but I do not know the details on how this would be done.

Yes, It could be a compute shader implementation but the current way they work requires a sync read back to CPU which invalidates their usage for anything happening each frame.

My scenario
I'm trying to generate triangles in the fly and render then each frame (It's for showing the 3d slice of 4d geometry made of tetrahedrons, but the exact use-case is not relevant). At least in theory should be possible using compute shaders to produce geometry by storing the result in buffers (generated triangle Indices and Vertex Data) and then rendering it without creating a Mesh.

I managed to get the compute shader part working, but no hopes in making the result go to the normal render pipeline like a normal Mesh would go, without actually reading the data back to CPU and creating the ArrayMesh. This basically turns compute shaders not useful for this kinds of tasks since the CPU syncs eats up any performance gains that we would otherwise gain.

In conclusion... If compute shaders were allowed to

• Write to Textures used by objects in the scene (Upcoming in 4.2 Add compute texture demo godot-demo-projects#938)
• Bind the same output buffer used in compute shader computations into a normal gdshader Add ability to pass compute buffers to vertex/fragment shaders #6989
• Dispatch a draw call with RenderingDevice buffers serving as the mesh data like the Unity's Graphics.DrawProceduralIndirect

Then they would be game-changing useful and allow things that truly harness the power of GPU's, like implementing compute shader based tesselation and virtual geometry.