Skip to content

HDR Output support#5962

Merged
FrancescoC-unity merged 68 commits into
masterfrom
HDRP/hdr-device-support
Oct 29, 2021
Merged

HDR Output support#5962
FrancescoC-unity merged 68 commits into
masterfrom
HDRP/hdr-device-support

Conversation

@FrancescoC-unity
Copy link
Copy Markdown
Contributor

NOTE: PR has lots of files that have lots of actual changes compressed by github, please when reviewing expand the diff.

This PR introduces proper HDR Output support for HDRP.

TL;DR of the features and characteristics:

  • Support for HDR10 and scRGB output standard.
  • ACES workflow for HDR (3 presets, 1000, 2000 and 4000 nits)
  • Adding Full ACES even for LDR workflow.
  • Neutral tonemapper using standard BT2390 curve for range reduction.
  • Added control over how much hue shift is desired.
  • Control over target min/max nits and display paper white.
  • Proper UI handling so that white in UI is maintained as white.
  • Added several debug visualizations, with special focus on gamut visualization.
  • Changed the whole grading pipeline to always use ACEScg as working space and allowing to go back to sRGB.
  • Optimized LUT building so it is created only when a change happens.

When enabling HDR all the HDR Output tunable parameters are shown in the Tonemapping volume component
image
image

Unfortunately cannot post images as they wouldn't show much on LDR screens and LDR screenshots :D The debug view can be seen here though

image
image
image

Most of the math for conversion and tone curves are in HDROutput.hlsl I tried to be verbose with comments there and added a bunch of desmos

A lot more can be written, but I am keeping this PR description tight. Please reach out and I am happy to discuss any aspect in more details.

I will be start writing docs now and will probably open a separate doc PR when I am done. Though I also need to find a way to fake a LDR/HDR comparison in LDR for Docs, and I have no clue how to do that :D

float slopeHigh; // log-log slope of high linear extension
};

half segmented_spline_c9_fwd(half x, SegmentedSplineParams_c9 params)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: we will need to mention this change in upgrade guide

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this because we expect external code to use this? It is trivial to add back the original version with the default in as before.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't know if users have used this function or not, so either we keep the old function, or as we expect it is a rarely used function, we inform the users in upgrade guide. But we should avoid to be silent when doing those kind of breaking change

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah to be safe I added the old signature/version so it is compatible and should work like before

Comment thread com.unity.render-pipelines.core/ShaderLibrary/ACES.hlsl Outdated
Comment thread com.unity.render-pipelines.core/ShaderLibrary/HDROutput.hlsl
Comment thread com.unity.render-pipelines.core/ShaderLibrary/HDROutput.hlsl
Comment thread com.unity.render-pipelines.core/ShaderLibrary/HDROutput.hlsl Outdated
Comment thread com.unity.render-pipelines.core/ShaderLibrary/HDROutput.hlsl
int hdrTonemapMode = m_Mode.value.intValue;
if (m_Mode.value.intValue == (int)TonemappingMode.Custom || hdrTonemapMode == (int)TonemappingMode.External)
{
EditorGUILayout.HelpBox("The selected tonmapping mode is not supported in HDR Output mode. Select a fallback mode.", MessageType.Warning);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo tonmapping

Comment thread com.unity.render-pipelines.high-definition/Runtime/Debug/DebugHDR.shader Outdated
#define _SizePerDim _HDRxyBufferDebugParams.xy
#define _IsRec709 (int)(_HDRxyBufferDebugParams.w == 1)

float2 RGBtoxy(float3 rgb)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same remark than for the other debug code, move to color.hlsl?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved at least partially

@kecho kecho self-requested a review October 25, 2021 13:12
Copy link
Copy Markdown
Contributor

@kecho kecho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dont see tests, are you planning to have some?

TextureHandle postProcessDest = RenderPostProcess(m_RenderGraph, prepassOutput, colorBuffer, backBuffer, cullingResults, hdCamera);

TextureHandle afterPostProcessBuffer = RenderAfterPostProcessObjects(m_RenderGraph, hdCamera, cullingResults, prepassOutput);
TextureHandle postProcessDest = RenderPostProcess(m_RenderGraph, prepassOutput, colorBuffer, backBuffer, uiBuffer, afterPostProcessBuffer, cullingResults, hdCamera);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I understand this. After FinalPass.shader we output a buffer in the OETF space. Looks like after you also apply (which is for dev only) again the signal to OETF and again apply UI.
I guess in dev mode we do the blit in a separate shader? Its just very confusing how its written. Perhaps we should be more explicit, such has having an enum with the schedule of the time we apply OETF. APPLY_OETF_IN_FINALPOST == OETF_APPLICATION_TIME etc etc.
This might make it more obvious. Otherwise I cant tell whats going on.

Another silly question, after our final frame is there, what takes care of the swap chain format? I dnt see any code that takes care of say HDR10 / etc on swap chain. Is this all already handled internally in the backend?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I understand this. After FinalPass.shader we output a buffer in the OETF space. Looks like after you also apply (which is for dev only) again the signal to OETF and again apply UI.

Nope. There is an already fairly explicit PostProcessIsFinalPass(HDCamera hdCamera) . The OETF by very definition needs to always be applied as the final pass, so it would be redundant to add an extra field that just says that.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another silly question, after our final frame is there, what takes care of the swap chain format? I dnt see any code that takes care of say HDR10 / etc on swap chain. Is this all already handled internally in the backend?

Yes, handled engine side.

@kecho
Copy link
Copy Markdown
Contributor

kecho commented Oct 27, 2021

For HDR screenshot comparison, what you can do is do something like bracketing. Take your frame buffer, and compare against a few stops. You will likely have to write a small test component. You could also do a histogram comparison instead, but I think comparing image to image will be tricky. What we want to test is the sanity of the pipe. Thoughts?

@FrancescoC-unity
Copy link
Copy Markdown
Contributor Author

Of course adding tests makes no sense :-) No HDR screen will be connected and no eye ballswill be there to look at HDR output anyhow.
To have something testable would be so artificial that it will be testing the test more than the actual output. Also it depends on the output device a lot so unlikely reproduceable.

This is just something better handled with manual testing.

@FrancescoC-unity
Copy link
Copy Markdown
Contributor Author

We could setup a whole exr capture grab and comparison, but it is way too overkill. We might consider that later on.

Copy link
Copy Markdown
Contributor

@kecho kecho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok we had a long discussion about auto testing. I will make a jira to address / discuss as a team the testing of this feature.
I still think that manual testing for HDR is not appropiate, but thats just my opinion :)

Copy link
Copy Markdown
Contributor

@JMargevics JMargevics left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello, made a thorough pass, things tested:

  • All Tonemapping modes
  • Debug modes
  • Runtime script to toggle HDR mode
  • All Windows platforms + PS4 Pro

More details in Conflunece doc

All good, and currently no issues from my side. Many thanks for making such a detailed and complete PR 👍✔

# Conflicts:
#	TestProjects/VisualEffectGraph_HDRP/Assets/HDRPDefaultResources/HDRenderPipelineGlobalSettings.asset
@FrancescoC-unity FrancescoC-unity merged commit be19220 into master Oct 29, 2021
@FrancescoC-unity FrancescoC-unity deleted the HDRP/hdr-device-support branch October 29, 2021 08:10
@FrancescoC-unity
Copy link
Copy Markdown
Contributor Author

The smoke test fail is known and reported, OSX passed upon re-runs https://unity-ci.cds.internal.unity3d.com/job/9655035

Merging.

unity-emilk pushed a commit that referenced this pull request Dec 16, 2022
POI: https://jira.unity3d.com/browse/POI-512

This PR introduces support for HDR Output to URP.
This includes HDR10 and scRGB output standards, bringing parity with HDRP. Tonemapping was extended when HDR output is active to include more control: 3 new ACES workflow presets (1k, 2k and 10k nits), range reduction mode on neutral tonemapper with or without hue shift, control over target min/max nits and paper white brightness.

When outputting to HDR, overlay UI needs to be handled differently so that white is maintained as paper white. The new pass `m_DrawOffscreenUIPass` takes the rendering of overlay canvas away from the engine and into an offscreen render texture, before post processing is applied. Since compositing the UI needs handling with the color encoding, and that color encoding needs to happen in the last pass, the UI texture is passed to multiple passes but only applied if that pass encodes the color.
If HDR output is not enabled but overlay UI is to be rendered by URP, then there's a new pass `m_DrawOverlayUIPass` happening after everything that renders directly onto the backbuffer/current target texture. This keeps the behaviour similar to when the engine renders it.

The [previous HDRP implementation](#5962) has also been reworked to align with URP (using the common `HDROutputUtils`), and to allow for future color gamuts to be added more easily. The most notable change is the keywords separated into `HDR_COLORSPACE_XXX` (color primairies & white point) that determines the colorspace rotation, and `HDR_ENCODING_XXX` (transfer function). This also adds `ColorGamutUtility` helper functions to define more clearly the `ColorGamut` reported by the display/platform.

Other additions:
- Warning in the editor UI have been added for incompatible features, as well as UX changes for some platforms (See [this document](https://docs.google.com/document/d/1BN06A8254NNv-OgQD0rC-6424AFIBERL9XllMqlFg6A/edit#) for visuals and details.)
- Internal mode to fake HDR on devices not connected to an HDR display, needed for automated testing.
- Platform-specific fixes
    - Vulkan now uses the 10-bit or 16-bit depth selected in Player Settings. 
    - Changed the bit depth name in Player settings so that it doesn't say D3D and added automatic upgrade for scripts to use the new enum by replacing the old one.
    - Changed the way we tonemap the game view window. We used to extract the title bar and tonemap it, but changed it to tonemap upfront before the gameview window is copied into it so that we can avoid all the code to extract out the titlebar from the window.
- Use SDR swapchain for URP scene camera (HDRP scene camera already uses SDR swapchain). The code to use HDR swapchain for Scene window is a little more involved and will try to tackle it in a later PR. It wasn't part of this Jira item and was a stretch goal.

Automated testing and documentation will land separately.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants