WebGPURenderer: Transmission - Backdrop 3/3

[sunag]

Related issue: #25903, #26196, #27850

Introduction

material.backdropNode allowed the manipulation of rendered objects as layers, similar to what we can do in Photoshop, for example, to add a node to overlay or any other filter with objects from the back, simply add material.backdropNode = viewportSharedTexture().overlay( someNode ) .

Here some exampels - https://threejs.org/examples/?q=backdrop

This PR introduces the last part of the backdrop implementation. This required some significant changes to the architecture, more details in the schematic below.

Schema

WebGPURenderer - New

WebGLRenderer - Current

Now all rendering is done on linear-colorspace, and only if color space conversion or tone mapping is needed, a simple post-processing layer is applied to the final rendering. This approach honors the settings defined in rendering regardless of whether the material is set to material.toneMapped=false or material.colorSpaced=false.

Refraction over Refraction

One of the big benefits is that we can have refraction over refraction, as we don't need RenderPass to backdrop work.

WebGPURenderer -> webgpu_loader_gltf_transmission_webgpu

WebGLRenderer -> webgl_loader_gltf_transmission_webgl

webgpu_loader_transmission

WebGPURenderer	WebGLRenderer

• Pros
  • Allows much more flexible management of refraction.
  • No need to render opaque objects twice, which can improve performance in large scenes.
  • -- Allows refraction over refraction. -- >> for other PR
  • Simplifies Shader per object, improves compilation and helps with caching.
• Cons
  • material.toneMapped=false and material.colorSpaced=false are no longer supported, these settings must be defined in the renderer.
  • An additional drawcall, although this is of little significance as it is a QuadMesh.
  • A little more gpu memory in some cases, but nothing to worry about.
  • Currently it's use copy buffer texture but it's possible use ping-pong approach to achieve full performance.

[mrdoob]

Looking good!

Do you mind adding stats.js?
The WebGPU version seems to be much slower than the WebGL version 🤔

[RenaudRohlinger]

@mrdoob on that not I'm working to add GPU performance monitor as an additional panel via renderer.trackTimestamp. It would be definitely useful for this example. Will be a major improvement for performance analytics in regard to WebGPU vs WebGL!

Related: #27870 #27597

[mrdoob]

Another thing I noticed is that the transmission seems to be one frame behind? (top-left sphere show the issue clearly)

Screen.Recording.2024-03-14.at.2.37.11.PM.mov

[mrdoob]

Let me know when this is ready to be reviewed again. The links in the first post no longer work.

[sunag]

Let me know when this is ready to be reviewed again. The links in the first post no longer work.

Yes! I think we can review it again.

[sunag]

New links... I removed the old examples but I can recovery if necessary

these are the new
https://raw.githack.com/sunag/three.js/dev-postprocessing/examples/index.html?q=webgpu%20transmission#webgpu_loader_gltf_transmission

[Mugen87]

I've noticed a minor thing in webgpu_materials_transmission. Compared to webgl_materials_physical_transmission, it seems the reflections of the inner faces are not visible through the transmissive front side. It's a bit hard to explain but if you open both examples and compare the refraction on the sphere, some details in webgpu_materials_transmission are missing.

[mrdoob]

@sunag

New links... I removed the old examples but I can recovery if necessary

Yes please 🙏

[sunag]

@Mugen87 I think that this is the same problem mentioned here #27880 (comment) referring to DoubleSided,

I think it's better to do this in another PR, this one is already very large, and makes very significant changes to the engine.

Explaining better this #27880 (comment) comment, to continue the agonotic renderer, the Renderer needs to have all the necessary resources so that we do not execute this process in core, for example in WebGLRenderer there is a very specific double pass when transmission have DoubleSided materials activated, where all transmission materials are rendered on the back-sided already with mipmaps applied of opaque materials, and after back-sided be rendered the mipmaps is generated again and render all transmission materials on the front-sided allowing you to see one face inside the other, this is possible following this approach, but I believe that due to the scope of the update it is better to leave it in another PR , and this will only really be necessary if a new refraction over refraction approach does not meet the performance needs.

[sunag]

@mrdoob The same code was used, just this change based from url parameter: renderer = webgpu ? new WebGPURenderer() : new WebGLRenderer()

WebGPURenderer with WebGLBackend fallback
https://raw.githack.com/sunag/three.js/dev-postprocessing-examples/examples/transmission.html?renderer=webgpu

WebGLRenderer
https://raw.githack.com/sunag/three.js/dev-postprocessing-examples/examples/transmission.html?renderer=webgl

[sunag]

@mrdoob I think performance will no longer be a problem, I just did some tests on the M1 in Chrome with the frame-rate-limit disabled and got an average of 172fps with WebGPURenderer vs 83fps using WebGLRenderer, this difference should increase if the scene had more opaque materials.

Only the points mentioned above that I must resolve in other PRs.

[mrdoob]

Looks good!