-
Notifications
You must be signed in to change notification settings - Fork 855
TAA and Physically based DoF #1193
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Haven't had a look at it yet (will try before vacation) but couple of discussion points:
Only discussions here, as I said haven't had the time to read the code yet :D |
Yes, I have tried DoF before TAA. On top of that, I have also tried to apply a simple TAA only on the CoC buffer. These were not enough to give a stable image.
I don't have any filtering/denoising (yet) because this is mostly for film/verticals use, and I think using high enough samples should be enough. But adding a filterng/denoising should definitely help. Maybe a future improvement.
Yes, a cheaper TAA before DoF is definitely a possibility, but I'm not aware what customizations are available for TAA. When you have time let's have a call on this. |
Btw, I have found a few issues when testing with the Amalienborg Palace, I'm working on an update. |
?? camera.AllocHistoryFrameRT((int)HDCameraFrameHistoryType.TAAMotionVectorMagnitude, Allocator, 2); | ||
previous = camera.GetPreviousFrameRT((int)HDCameraFrameHistoryType.TAAMotionVectorMagnitude); | ||
int historyType = (int) (postDoF ? | ||
HDCameraFrameHistoryType.TAAMotionVectorMagnitude : HDCameraFrameHistoryType.TAAMotionVectorMagnitudePostDoF); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You don't need the post dof as we don't do speed rejection in the ad-hoc pass. No need to allocate another history buffer
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In fact we don't need to modify this function at all, just don't call it when you do your ad-hoc TAA pass
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good to know, will push the change (along with the other comments) soon.
?? camera.AllocHistoryFrameRT((int)HDCameraFrameHistoryType.TemporalAntialiasing, Allocator, 2); | ||
previous = camera.GetPreviousFrameRT((int)HDCameraFrameHistoryType.TemporalAntialiasing); | ||
int historyType = (int)(postDoF ? | ||
HDCameraFrameHistoryType.TemporalAntialiasing : HDCameraFrameHistoryType.TemporalAntialiasingPostDoF); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Kind of related, but this allocates effectively another history buffer, we are a bit struggling on memory at the moment, I am not sure if we can, but it'd be nice to deallocate the history if we had it allocated but we are not using postDoF anymore (Or alternatively, add an additional asset setting for this dof so that we never allocate this buffer if users doesn't need it)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(I would look for the API, but am doing the review in between things, I am sure there is a way to release an history target)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have added a clean-up call, but of course the cleanest and more general way to deal with this stuff is a render graph.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yup, but rendergraph at the moment still takes history from outside and imports them in so the issue would still be present.
Cool for the cleanup!
|
||
if (taaEnabled) | ||
{ | ||
bool useMips = true; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Question: is this exactly the same as the old coc reprojection code? If so can you please move it to a common function? If not can we make it so it can be in a common function with that?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Moved it to a common function (only difference with the old code is the use of mip maps). Keep in mind that we will need to improve this re-projection too in the future (it has some ghosting).
float linearEyeDepth = LinearEyeDepth(depth, _ZBufferParams); | ||
|
||
_OutputTexture[COORD_TEXTURE2D_X(dispatchThreadId.xy)] = 0.0f; | ||
const float FarStart = _Params.x; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's fine like this as compiler is happy anyway, but in general the pattern we use for this is defines straight after the cbuffer defintion (e.g. see for example the MotionBlurCommon.hlsl file)
float GetNumSamples(float radius, float maxRadius) | ||
{ | ||
//TODO: needs to take into account the density push towards the center (when enabled) | ||
// Note: should also take into account the density push towards the center (if enabled), this needs to be fixed at some pointss |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
small typo here (point vs pointss)
{ | ||
float currentAvg = RING_COLOR[2].w * rcp(ring * RING_DENSITY); | ||
float prevAvg = prevRingWeight[2] * ((ring > 1) ? rcp((ring - 1.0) * RING_DENSITY) : 1); | ||
float currentAvg = RING_COLOR[j].w * rcp(ring * RING_DENSITY);; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
double ; here
/// <summary>Temporal antialiasing history after DoF.</summary> | ||
TemporalAntialiasingPostDoF, | ||
/// <summary>Velocity magnitude history used for TAA velocity weighting (after DoF).</summary> | ||
TAAMotionVectorMagnitudePostDoF, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Again don't need this
/// <summary>Use Depth of field high quality filtering for each quality level.</summary> | ||
public bool[] DoFHighQualityFiltering = new bool[s_QualitySettingCount]; | ||
/// <summary>Use Depth of field high physically based setting for each quality level.</summary> | ||
public bool[] DoFPhysicallyBased = new bool[s_QualitySettingCount]; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Small alignment issue
I have not done a pass on the algorithmic part yet (might not be able to :/ ), but can try later this afternoon. If I don't, go ahead and I'll have a look after my vacation :-) |
} | ||
|
||
TemporalAntiAliasingParameters PrepareTAAParameters(HDCamera camera) | ||
TemporalAntiAliasingParameters PrepareTAAParameters(HDCamera camera, bool PostDOF = false) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lower case for the param in the function
RTHandle nextHistory, | ||
RTHandle prevMVLen, | ||
RTHandle nextMVLen) | ||
RTHandle prevMVLen = null, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not a big fun of the default parameter here, probably better to pass a null rt explicitly in the function call like
DoTemporalAntialiasing( ......, prevMVLen: null, nextMVLen: null) makes it way more explicit on caller side.
#define ANTI_FLICKER_MV_DEPENDENT 1 | ||
#define VELOCITY_REJECTION (defined(ENABLE_MV_REJECTION) && 0) | ||
#define PERCEPTUAL_SPACE 1 | ||
#define PERCEPTUAL_SPACE_ONLY_END 0 && (PERCEPTUAL_SPACE == 0) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Didn't we want to disable the sharpening too?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I had a version with the post-dof sharpening disabled, but in the end it is not what we want (it was "reverting" the pre-dof sharpening).
The right thing to do is to perform sharpening only the last time TAA will run, but I might have to introduce a new variant to implement that (because it means that medium TAA might need to run without sharpening, depending on the DoF settings!). As it is, I don't get any artifacts (as long as we don't do two times the 5tap bicubic), so I left this as a low priority, until I resolve another bug.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
/// The minimum allowed aperture. | ||
/// </summary> | ||
public const float kMinAperture = 1f; | ||
public const float kMinAperture = 0.7f; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am almost certain it will work fine, but worth checking how changing this affects existing projects
/// <summary>Number of history buffers.</summary> | ||
/// <summary>Temporal antialiasing history after DoF.</summary> | ||
TemporalAntialiasingPostDoF, | ||
/// <summary>Velocity magnitude history used for TAA velocity weighting (after DoF).</summary> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Mismatched comment now :) Should be
/// <summary>Number of history buffers.</summary>
45434e6
to
3cb3a4e
Compare
Merging for the 10.x package. We will require to add an addition DOF test for it. In a next PR. + in the doc , be more explicit on "slower", i.E maybe we can say it is not recommended for game? can ask Lewis |
… and spot ray traced shadows (1261040). (#1285) * Modify scene for better stability (#1120) * Fix area light gizmo not updating engine side light v2 (#1156) * Update base light area size. * changelog * Better change. * revert whitespace * Make the change editor only. * Updated doc. (#1152) * Address decal layer documentation feedback (#1157) * Update Decal-Projector.md * update doc based on feedback * Hdrp/docs/upgrading to 2020.2 (#1170) * Update Upgrading-from-2020.1-to-2020.2.md * Update Upgrading-from-2020.1-to-2020.2.md * HDRP - Shutter speed can now be changed by dragging the mouse over the UI label (#1110) * Shutter speed can now change by dragging over the label * Changelog * Typo Co-authored-by: sebastienlagarde <sebastien@unity3d.com> * Fix API breaking change: Add Public quality to Raytracing volume effect - RR * ix API breaking change: Add Public quality to Raytracing volume effect - RR * Add 3 tests in HDRP_Test for 1709_DecalLayers for deferred, forward and forwardMSAA (#1138) * add 1709 test * More decal test * Update 1709_DecalLayer.unity * fix material * Update test * update test * Add 3 new test forward, forward MSAA and deferred * add the 3 test in build settings * Add reference screenshots for windows * update reference screenshots * disabe 1709 with rendergraph * Fixed creation of Decal Buffer (should not have been bindMS) * Fixed missing read causing copy depth pass to be pruned * Fixed clear color of decal prepass buffer * Re-enabled tests 1709 1710 and change order to workaround resolution issue when running tests with temporal. Co-authored-by: Julien Ignace <julien@unity3d.com> * Added what's new page and relevant images (#1154) * Added what's new page and relevant images * Update whats-new-10-0.md * Added decal layers and eye shader graph Co-authored-by: sebastienlagarde <sebastien@unity3d.com> * update what's new * Fix Material Creation RenderQueue setup + Fix AXF GUI (#1185) * Fix Material creation queue * Update SurfaceOptionUIBlock.cs * Refactor keyword system in ShaderGraph (#1181) * Refactor the keyword system in ShaderGraph * Move geometric AA to respective master node Co-authored-by: Sebastien Lagarde <sebastien@unity3d.com> * Made conversion from render graph handles to actual resource implicit. (#1034) * Made conversion from render graph handles to actual resource implicit to reduce boiler plate code. * Post merge fixes * Added an exception when accessing resources while not executing render graph. * Post merge fixes * Fixed error handling of casting handles to resources. * Fix double sided unlit mode in the UI (#1203) * HDRP: Default indirect Lighting controller Layers to Everything to ease the migration * Update cas include paths (#1227) * Hdrp/fix rt odd negative scale (#1180) * Fixed robustness issue with GetOddNegativeScale() in ray tracing. * Updated changelog. * Added fogbugz case id in changelog. * Hdrp/fix hierarchicalbox gizmo symetry and homothety mode ensuring face faces outside (#1228) * Fix hierarchicalbox gizmo outside facingin symetry or homothety mode * Update CHANGELOG.md * Update CHANGELOG.md * Update CHANGELOG.md * Update CHANGELOG.md Co-authored-by: sebastienlagarde <sebastien@unity3d.com> * fix XR single-pass macros in tessellation shaders (#1160) Co-authored-by: sebastienlagarde <sebastien@unity3d.com> * Remove automatic setup of global texture from Render Graph (#1192) * Made conversion from render graph handles to actual resource implicit to reduce boiler plate code. * Post merge fixes * Added an exception when accessing resources while not executing render graph. * Removed auto binding of global texture and updated passes accordingly. * Post merge fix * Fixed shadow mask texture binding. * Fix wrong merge * Fixed fog with MSAA and render graph * Hdrp/atlas cube modif (#1235) * Remove the usage of Point Cube Size parameter. * Add change log * Hdrp/pt sss diff mix (#1224) * Fixed robustness issue with GetOddNegativeScale() in ray tracing. * Updated changelog. * Added fogbugz case id in changelog. * Mofified way diffuse and SSS are mixed in path tracing. * Fixed issue with spec balance. * Updated Changelog. * Updated ref image for test 5007. * Change Cloud Layer API (#1237) * Change cloud layer API * fix line endings * Change default value * ShaderGraph UI fixes (#1216) * Save foldout state in ShaderGraph * Moved all non-SG properties to the advanced UI block * Reordered advanced properties * Move shadergraph properties (#1214) * Migration code for depth pre/post pass and lod crossfade in shadergraph * Removed log * Added "first time" migration system * Fix custom pass reorder (#1153) * Fixed custom pass re-ordering issues * Updated changelog Co-authored-by: sebastienlagarde <sebastien@unity3d.com> * Revert "Move shadergraph properties (#1214)" This reverts commit 9a8d3b1. * Update Upgrading-from-2020.1-to-2020.2.md * Make sure we don't end up with null normals from normal mapping, when scale is 0. (#1233) * Make sure we don't end up with null normals from normal mapping. * Updated Changelog. Co-authored-by: sebastienlagarde <sebastien@unity3d.com> * Contact shadows improvements (#1226) * Bias + thickness + new depth compare * Default as decided by pierre * upgrade guide * docs * changelog * What's new * Revert "Revert "Move shadergraph properties (#1214)"" This reverts commit 958486d. * Fix migration issue in PR (Move shadergraph properties #1214) * TAA and Physically based DoF (#1193) * stabilize CoC if taa is enabled * Fixes related to CoC clamping * Improve DoF when TAA is active by using different samples each frame * Add UI option to run the new physically based DoF * Update documentation with the new option * Fix issue with mip maps when switching DoF techniques * Improve documentation * Clean-up C# code * Minor changes in the shader * Fix near-far blending and cleanup unused code * clean-up shader code * more shader shader clean-up + fix weigths * more shader shader clean-up * back-to-front blending clean-up * Make Manual CoC work for the new dof * Changelog * Change min aperture to 0.7 * Add info box when physically based dof is enabled * Add info box when physically based dof is enabled * Add s specialized TAA configuration for use with DoF and also stabilize CoC * Do not increase sample count when taa is off * Fix issue in disc sampling * Fix profiling scope for CoC * Fix for adaptive search radius * Review feadback * Review feadback 2 * Fix non-uniform ring weights * Fix an issue with DofGather.compute not compiling on Metal * DoF: avoid calling GetDimensions for LoDs in the shader (#1256) * DoF: Pass mipmap dimensions as shader constant * DoF: invalidate TAA history when switching modes * convert Material from ~samples to latest version. (#1297) * Update Vulkan reference images for tests 9601 and 9602 (#1158) * remove reference images for 9601 and 9602 on Vulkan WindowsEditor to force regenerate them * update reference images (9601, 9602) from Yamato artifacts for Vulkan * fix shader macros for tessellation and XR single-pass * restore shader changes (moved to another branch) * HDRP - Update item menu for Decal layers after doc team feedback * Added documentation for material GUI (#1082) * Started to move as public a part of the material GUI * Moved HDUnlitGUI to public * Moved back to internal * internal * Fixed some doc * Fixed API doc * Fix API doc * Add missing doc * Updated doc Co-authored-by: sebastienlagarde <sebastien@unity3d.com> * Various small fixes for render graph. (#1232) * Made conversion from render graph handles to actual resource implicit to reduce boiler plate code. * Post merge fixes * Added an exception when accessing resources while not executing render graph. * Removed auto binding of global texture and updated passes accordingly. * Post merge fix * Fixed shadow mask texture binding. * Fix wrong merge * Fixed fog with MSAA and render graph * Replace new TextureHandle by TextureHandle.nullHandle * Fixed name of some textures * Implemented stub of ScreenSpaceShadows to fix default texture binding. * Correctly deallocate cached shadows atlas * Properly set color buffer for transparent custom passes. * Small comment update. * Made naming of resources created by RG generic (user names are used for render graph specific debug display) * Fixed custom pass global texture binding. * Added name to real time reflection probe textures. * Added an option to disable pass pruning and simplified logging code. * Added XR copy depth * Rename "pass pruning" to "pass culling" * Fixed tests compilation * Revert logger changes because they made unavoidable allocs. * Removed useless formatting. * Lighting/probevolumes debug modes and uniform init cleanup (#1275) * Probe Volumes: Cleanup DebugDisplayProbeVolume and support SH2 atlas display. * Probe Volumes: Bugfixes to state when Probe Volumes is enabled in Shader Config, bug disabled in the HDRenderPipelineAsset or the current cameras frame settings. * Only building the RTAS if there is an effect that will require it (1262217). (#1278) * Only building the RTAS if there is an effect that will require it (1262217). * missing screenshot * Fixed rendering breaking when disabling tonemapping in the frame settings. (#1284) * Fixed rendering breaking when disabling tonemapping in the frame settings. * Update changelog. * Fixed a null ref in the volume component list when there is no volume components in the project (#1282) * Fixed a null ref in the volume component list when there is no volume components in the project * Update changelog * Fixed changelog * Make sure serialization of exposure modes is consistent between HDRP versions (#1286) * Make sure serialization of exposure modes is consistent between HDRP versions * Fix visual ordering + defaults for new scenes Co-authored-by: sebastienlagarde <sebastien@unity3d.com> * Avoid using duplicate names when creating new sublayers in the graphics compositor (#1288) Co-authored-by: sebastienlagarde <sebastien@unity3d.com> * Remove MSAA debug mode when renderpipeline asset has no MSAA (#1289) Co-authored-by: sebastienlagarde <sebastien@unity3d.com> * Fixed an issue where look dev lighting would go black when a new scene is loaded. (#1303) * Fixed some post processing using motion vectors when they are disabled (#1305) Co-authored-by: sebastienlagarde <sebastien@unity3d.com> * add playmode_XR to HDRP Vulkan tests (#1306) * Fixing nans related to reflection probes in ray tracing and potentially instabilities. (#1307) * - Fixed the multiplier of the environement lights being overriden with a wrong value for ray tracing (1260311). - Fixed the first ray tracing frame not having the light cluster being set up properly (1260311). * Added docs. * Update property format in doc entries Co-authored-by: Lewis Jordan <lewis.jordan@hotmail.co.uk> Co-authored-by: sebastienlagarde <sebastien@unity3d.com> * Small fixes (#1309) * Fixed nullref exception with cookies * Fixed GC.Alloc * Fixed a series of exceptions happening when trying to load an asset during wizard execution (1262171). (#1312) Co-authored-by: sebastienlagarde <sebastien@unity3d.com> * Render graph pre-setup for ray traced ambient occlusion. (#1026) * - Render graph pre-setup for ray traced ambient occlusion. * Review corrections Co-authored-by: sebastienlagarde <sebastien@unity3d.com> * Fix Shader compilation errror in DEbugDisplayProbeVolume.shader * Fix exposure settings in Volume Settings for standalone * Avoid casting multiple rays and denoising for hard directional, point and spot ray traced shadows (1261040). Co-authored-by: remi-chapelain <57442369+remi-chapelain@users.noreply.github.com> Co-authored-by: FrancescoC-unity <43168857+FrancescoC-unity@users.noreply.github.com> Co-authored-by: Emmanuel Turquin <emmanuel@turquin.org> Co-authored-by: sebastienlagarde <sebastien@unity3d.com> Co-authored-by: JordanL8 <lewis.jordan@hotmail.co.uk> Co-authored-by: Pavlos Mavridis <pavlos.mavridis@unity3d.com> Co-authored-by: Julien Ignace <julien@unity3d.com> Co-authored-by: Antoine Lelievre <antoinel@unity3d.com> Co-authored-by: Remi Slysz <40034005+RSlysz@users.noreply.github.com> Co-authored-by: Fabien Houlmann <44069206+fabien-unity@users.noreply.github.com> Co-authored-by: skhiat <55133890+skhiat@users.noreply.github.com> Co-authored-by: Adrien de Tocqueville <adrien.tocqueville@unity3d.com> Co-authored-by: Nicholas Brancaccio <pastasfuture@gmail.com>
Purpose of this PR
This PR adds a new UI option in the DoF volume in order to enable the physically-based DoF that was previously merged.

More importantly, it improves the quality (and stability) of DoF when TAA is active by using different sample positions on every frame. TAA is executed two times, one before DoF to stabilize the input to the DoF algorithm, and one after DoF, to accumulate aperture samples over multiple frames. This PR also removes unused code paths from the shader.
In practice, similar to some other effects in HDRP, enabling TAA is highly recommended to get a clean image.
Here is a comparison with the physically-based mode ON and OFF:
Physically-based ON:

Physically-based OFF:

The red arrows show the main areas of improvement of the physically-based method:
The code right now is not optimized at all, so there is a lot of low hanging fruit to make it faster.
Testing status
The new DoF mode is inactive by default, so it should not break existing projects. It will only affect users that explicitly enable it.
Note: It happens very rarely, but in some scenes I have noticed some occasional flickering at the edges of the frame (probably related to the TAA settings). I'm investigating this and will update with a fix.