Examples: Add WebGPU HDR Example

[RenaudRohlinger]

Related issue: #29573 (comment)

Description
Add a basic example to test and demonstrate Extended SRGB ColorSpace via High Dynamic Range in WebGPU.

Heavily inspired by @greggman's HDR demo https://github.com/greggman/HDR-draw

Example:
https://raw.githack.com/renaudrohlinger/three.js/utsubo/feat/hdr-example/examples/webgpu_hdr.html

IMG_1211.MOV

This contribution is funded by Utsubo

[Mugen87]

Can we then remove the HDR usage from webgpu_tsl_vfx_linkedparticles now that we have a dedicated example?

[donmccurdy]

@RenaudRohlinger The working color space will need to remain Linear-sRGB (the default value). If this causes other issues I think we can work that out! You're correct that we do need to register Extended sRGB and assign it as the output color space, though.

Conceptually, you can think of Extended sRGB as a color space that pre-supposes some fixed representation of "white" relative to the display, and then provides a mechanism for output at greater wattage than this "white". There are technical/conceptual/perceptual issues with that approach, but it's what WebGPU HDR gives us today, so we'll use it as the output space.

In our working color space, prior to image formation (exposure, tone mapping), we can think of the RGB values as stimulus received by the camera. There's no image yet formed, nothing is relative to any display or viewing environment, and without such a reference to anchor perceptual interpretation there is no "white". So Extended sRGB has no practical meaning as a working color space in a lit rendering pipeline.

Additionally, Extended sRGB uses a non-linear transfer encoding that isn't valid for the PBR rendering process.

[RenaudRohlinger]

I see, thanks for the great explanations @donmccurdy!

[RenaudRohlinger]

Thanks for the reviews, should be good now!

[WestLangley]

@RenaudRohlinger How would you feel about renaming this example to webgpu_extended_srgb.html ?

[RenaudRohlinger]

That’s definitely more pertinent, but I’m not sure the concept of extended_srgb is very accessible to most people. In practice, many developers would probably expect to find this under HDR. If we do this, we could address this by using tags.json to alias the search with HDR.

[WestLangley]

Well, that is what the demo is about.... I think your tags suggestion is a good idea.

[donmccurdy]

My feeling is that the "Extended sRGB" space is a necessary evil to access the capabilities of the display at this point — if WebGPU were to expose Rec 2100 HLG (or similar) I would prefer to switch the example over to that as the output color space.

[WestLangley]

I have some questions about this example. (I have a silicon iMac.)

1. renderer.outputColorSpace = ExtendedSRGBColorSpace - Commenting this out does not appear to have any effect. I also question the API here. I think the output color space should be the linear working color space in this case.

2. It appears the srgb OETF (gamma) is being applied by the shader. My understanding is it should not be. The system applies tone mapping and gamma in this case. Granted, if the OETF is not applied by the shader, the rendering is too dark, so something is not correct...

3. I see no evidence of tone mapping being applied -- just increased brightness. How can we ensure this is working as intended?

4. In the WebGPUBackend, context.configure() does not set color space, leaving it at the default sRGB. I think the color space should be consistent with the working color space, which can be linear display-p3, for example.

/ping @RenaudRohlinger @donmccurdy @sunag

[donmccurdy]

1. renderer.outputColorSpace = ExtendedSRGBColorSpace ...

Ah, this line is incorrect:

three.js/src/renderers/webgpu/WebGPUBackend.js

Line 258 in 3220e98

const toneMappingMode = parameters.outputType === HalfFloatType ? 'extended' : 'standard';

Instead we want to be checking the color space's defined tone mapping mode...

three.js/examples/jsm/math/ColorSpaces.js

Line 133 in 3220e98

outputColorSpaceConfig: { drawingBufferColorSpace: SRGBColorSpace, toneMappingMode: 'extended' }

... so that use of outputType=HalfFloat does not send output beyond [0,1] as it does here.

2. It appears the srgb OETF (gamma) is being applied by the shader...

This how "Extended sRGB" is defined, and so unfortunately necessary under the current WebGPU HDR spec. Similar to my comment above, I think this is a problematic choice in the spec, and I'd be glad to drop support for Extended sRGB given any other option. It is, indeed, a non-linear sRGB encoding on the [0, ∞] domain.

3. I see no evidence of tone mapping being applied...

I believe there are two mechanisms operating under the name "tone mapping" here, confusingly. The first is three.js' own tone mapping, which we haven't yet implemented for HDR output. I think it's important that we add that, even if the Extended sRGB output space is not sufficient for us to design it correctly, if only to understand the limitations better.

The second is WebGPU's (or the OS's?) own "tone mapping" to adapt the formed image to the display. That behavior is visible in this example in the form of increasing/decreasing contrast when changing the brightness of the laptop display on macOS. Available HDR "headroom" decreases as the laptop brightness increases.

[donmccurdy]

Hm, if I set a breakpoint at WebGPUBackend.js#L258, I'm seeing this.renderer.outputColorspace = 'srgb-linear' even though it's configured to ExtendedSRGBColorSpace. I assume this is related to post-processing, but I'm not sure how to detect the final output space in order to configure the canvas correctly...

[sunag]

We don't have an ExtendedLinearSRGBColorSpace for workingColorSpace, so in some processes the context.configure() initialization process may be being triggered in LinearSRGBColorSpace, this is due to post-processing since the color output transformation is done in the RenderOutputNode where context.configure() has already been called.

About the concept of extended, the API seems very declarative to me; I think Three.js should make things easier for this.

Using outputType: THREE.HalfFloatType in an output without HDR should be seen as a sign of poor optimization on the user’s part, so if outputType = THREE.HalfFloatType, it should already be classified as extended sRGB or equivalent according to the color gamut, and that should be handled internally. Currently in any scenario where HDR isn’t available, it’s currently up to the user to implement all fallback modifications, when this could be automated by the renderer.

We could warn when a tone mapping and output color space configuration isn’t compatible with HDR instead of recreating an public Extended* one by one. That would be simple to implement in ColorSpaceNode and RenderOutputNode.