Support AgX tone mapping

[donmccurdy]

Description

I think we should offer AgX as a tone mapper in three.js. It has better fundamentals than others like Reinhardt or ACES¹, often provides a more natural appearance, and — importantly — is the new default in Blender 4.0. Supporting AgX would make consistent color across Blender and three.js much easier for users.

Additionally, I feel more comfortable using AgX in wide-gamut workflows (see #26479).

Solution

AgX has been discussed in a few other threads previously:

• Roadmap for supporting wide-gamut color workflows #26479 (comment)
• Pluggable Tone Mapping Operators (their own class that injects GLSL) #26661 (comment)

While I've done some proof-of-concept implementations myself, none are ready for production. Notably, we'd want to replace the LUTs with a piecewise-linear function (or similar), and perhaps to support the "golden" and "punchy" looks.

The Filament implementation is probably a good reference here.

Alternatives

• "Filmic": Previous Blender default. AgX improves on this, and I'm not sure there's any particular reason to go back to Filmic.
• "Commerce": Available in model-viewer, and has starts from somewhat different goals than AgX. I believe AgX is probably the place we should start, but Commerce could be a worthwhile addition later.

Additional context

No response

Footnotes

1. I don't meant to say that users can't or shouldn't use ACES or Reinhardt. If you prefer the appearance of ACES for some specific scene, that's completely fine. I do mean to say that AgX is a better default, and a better starting point from which to begin making artistic choices (like other tone mapping) when creating a new project. ↩

[gkjohnson]

I'd very much like to see this - here are a couple sources for implementation:

• Here's an article that includes source and a link to a shader toy implementation
• And a fork of the original AGX repo which includes LUTs that could be used with the LUTPass postprocessing effect if I understand correctly.

[Mugen87]

Maybe we can start with a new function AgXToneMapping() in <tonemapping_pars_fragment> and the respective constant THREE.AgXToneMapping so we can use it inline and in OutputPass?

[donmccurdy]

@gkjohnson huge thanks and great work getting AgX done for r160! I'm very excited for this. :)

I'm inclined to go through some of our existing examples that are 'lit' but either (a) don't currently use tone mapping, or (b) use ACES Filmic, and try switching some to AgX. Let's see how it goes; I have a hunch we may want the contrast control of AgX 'looks'. I am wondering if perhaps that should be a separate property from the tone map, like:

renderer.toneMapping = THREE.AgXToneMapping;
renderer.toneMappingLook = THREE.AgXHighContrastLook;
renderer.toneMappingExposure = 1.5;

If we wanted (in the future) to allow custom 'looks' without replacing the entire tone mapping, that would be an easier option this way.

[gkjohnson]

If "looks" are going to be added I'd like to understand what the value or intent of using them are over color grading the AgX-transformed image (via something like custom contrast controls or LUTs). It seems like we want to use AgX "Looks" in lieu of a more powerful color grading step in three.

renderer.toneMapping = THREE.AgXToneMapping;
renderer.toneMappingLook = THREE.AgXHighContrastLook;

If we do wind up adding the looks I think I'd prefer them to be added as THREE.AgXPunchyToneMapping etc. Adding another field that only works when AgX is used will be confusing, I think. Ie what happens in this case:

renderer.toneMapping = THREE.ACESToneMapping;
renderer.toneMappingLook = THREE.AgXHighContrastLook;

[kitaedesigns]

I think for 3D artists like myself having the ability to use a LUT would be quite natural also easier to fine-tune the look of a scene using assets we probably already have.

If "looks" are going to be added I'd like to understand what the value or intent of using them are over color grading the AgX-transformed image (via something like custom contrast controls or LUTs). It seems like we want to use AgX "Looks" in lieu of a more powerful color grading step in three.

renderer.toneMapping = THREE.AgXToneMapping;

renderer.toneMappingLook = THREE.AgXHighContrastLook;

If we do wind up adding the looks I think I'd prefer them to be added as THREE.AgXPunchyToneMapping etc. Adding another field that only works when AgX is used will be confusing, I think. Ie what happens in this case:

renderer.toneMapping = THREE.ACESToneMapping;

renderer.toneMappingLook = THREE.AgXHighContrastLook;

[donmccurdy]

Unorganized thoughts:

• Personally I feel that throwing an error for an incompatible Tone Mapping and Look would be unambiguous. The Look is a component of the tone mapping, and meaningless outside of that context, and we can word the error accordingly
• Blender includes nine "looks" for AgX, mostly variations in contrast. There's also the "Use Curves" custom panel, which isn't portable, is more difficult to use, and more prone to user error
• I'm also fine with something like THREE.AgXPunchyToneMapping
• It's unclear to me whether contrast-modifying Looks are easily replaced with a floating point value, but if so, .toneMappingContrast = +1.0 could be an alternative
• It's likely significant that the Look is applied before the AgX outset matrix
• I suspect that full-fledged color grading APIs belong in the post-processing module, or perhaps in nodes, and not in the core shader chunks. Same for LUTs.
• The postprocessing package already provides general-purpose, composable color grading effects.
• I feel that users often misunderstand or ignore the value of tone mapping, or think of it as they might think of an Instagram filter. Our current tone mappers contribute to that misunderstanding, whereas I think AgX sets a better and more neutral baseline, which could potentially be our default. But the benefit to users is best if they can use it without being forced into a full color grading API. It'd feel unfortunate for a small tweak to AgX contrast to require addition of a post-processing pass.

[kitaedesigns]

Hi all, maybe this is what you were talking about with reference to "AgXPunchyToneMapping" but I am noticing some discrepancies between Agx in Blender vs threejs which I have also posted in pmndrs/postprocessing.
pmndrs/postprocessing#555 (comment)

Here is a comparison with threejs and blender's AgX. Blender on the Left. Threejs on the Right. Using the exact same asset and cubemap.

Blender AgX with Exposure at 0 vs Threejs AgX with Exposure at 1.

If I manually adjust each one's exposure I can get them quite close but I think the expectation would be that Exposure of 1 with AgX tonemapping in Threejs should be the same as Exposure 0 AgX tonemapping in Blender.

Here is an example with a Baked Lighting model which a user would expect to look exactly the same between platforms.

[donmccurdy]

@kitaedesigns does exposure match up between the two when tone mapping is disabled? "Standard" view transform in Blender terminology. I expect not — the entire rendering pipeline is different — and AgX cannot correct for that.

I believe a correct mapping of exposure in Blender to exposure in three.js exists, but is not as simple as 0 => 1.

For the baked lighting: generally one does not tone map after baking to an unlit material. Tone mapping should have open domain [0, infinity] input, and baking to a PNG constrains input to [0, 1]. I'm not sure if Blender can bake the AgX view transform into the baked base color texture, but that would be one correct approach to solve this.

[kitaedesigns]

Ah I see, that's what I thought, the two rendering pipelines are totally different. I suppose the best thing to do is add some Color Grading, Hue, Saturation, Brightness, and possibly a LUT compressed with KTX that I manually create in Photoshop to try and match the two AgX environments as much as possible?

For Baked Lighting I am not using an Unlit material but plugging the baked lighting texture into the Diffuse and Emissive slots so I can still use the Metallic and Roughness channels to get some nice HDR reflections which is why I am using Tonemapping.

[donmccurdy]

I suppose the best thing to do is add some Color Grading, Hue, Saturation, Brightness, and possibly a LUT compressed with KTX that I manually create in Photoshop to try and match the two AgX environments as much as possible?

I believe the primary difference to correct for here is exposure — Blender uses different lighting units, and exposure can adjust the image formation process for that. Custom color grading beyond AgX is likely to take you farther away from Blender, if that's what you're trying to match. Conceptually three.js and Blender have implemented the same image formation — AgX — and it doesn't particularly matter whether that's a LUT or a shader.

Tone mapping on top of lighting baked into PNG or JPEG textures will diverge from Blender as well, unless you put the baked textures back into your material in Blender and preview tone mapping htere. It would likely be much simpler to test without mixing baking and tone mapping, then add it back once color matches your expectations.

[kitaedesigns]

Hmm I think I will test with a Color Calibration model https://1drv.ms/u/s!AksR0p9ftT9YjjSMUE5lEfZvyH2N?e=11EL4c and try and match AgX at Exposure 0 in Blender with AgX Exposure 1 in ThreeJS in Photoshop and try and make a LUT that corrects for the differences.

[gkjohnson]

try and match AgX at Exposure 0 in Blender with AgX Exposure 1 in ThreeJS in Photoshop and try and make a LUT that corrects for the differences.

See the thread in #27366. The tonemapping in Blender is more complex than can be captured in a LUT as it at least adjusts for negative values across the entire image.

[donmccurdy]

I will be curious what results you find here. My belief is that exposure=0 in Blender is fundamentally not the same thing as exposure=1 in three.js and that trying to align them with a LUT will be more roundabout and fragile than trying to adjust the exposure directly.

---

@gkjohnson in the interest of keeping the API as-is for now, what do you think of just adding the "Punchy" look as an additional tone mapping enum type? I think we can defer on adding the various levels of contrast for now.

[gkjohnson]

in the interest of keeping the API as-is for now, what do you think of just adding the "Punchy" look as an additional tone mapping enum type? I think we can defer on adding the various levels of contrast for now.

Sounds fine to me!

[kitaedesigns]

When I was suggesting making a LUT I was initially thinking about making it so it would match Blender Exposure 0 with Threejs Exposure 1, this should be feasible, but yes if you were to change the exposure values they would be more different due to how each one calculates Exposure.

[donmccurdy]

PR for AgXPunchyToneMapping —

• WebGLRenderer: Add 'AgXPunchyToneMapping' #27618