TAA off screen reprojection FXAA fallback #8847
Conversation
alice-i-cecile
added
A-Rendering
Co-authored-by: JMS55 <47158642+JMS55@users.noreply.github.com>
| // Compare current and previous depth | ||
| let current_depth = 1.0 / current_depth_nonlinear; | ||
| let last_frame_depth = 1.0 / textureSampleLevel(last_frame_depth, nearest_sampler, history_uv, 0.0); | ||
| if abs(current_depth - last_frame_depth) > 0.05 { |
There was a problem hiding this comment.
@JMS55 I don't think we should use the linear depth here. It will cause TAA to act very differently depending on how close the camera is to things. Things far in the distance will now be much more likely to trigger disocclusion. I think we should probably use the pixel radius, or non linear depth somehow. I haven't looked yet at what is typically used for TAA.
I noticed in this version that there's a lot more aliasing under motion than before on the edges of the boxes as well as on the noisy plane.
There was a problem hiding this comment.
Typically you check for disocclusion by checking if the difference in depth is greater than ~10%. I thought the depth of the texture was already non-linear. We should probably be using world/view space depth. I always forget what the depth texture is stored as though... (we could really use some docs about the prepass formats).
There's also a plane distance check that has less positive disocclusions, but I've left that for future work.
I noticed in this version that there's a lot more aliasing under motion than before on the edges of the boxes as well as on the noisy plane.
Can you try plugging in closest_depth instead of d_mm into the disocclusion checks and see if that fixes it?
There was a problem hiding this comment.
The texture is non-linear but you convert it to linear by dividing it by 1.0. (for linear view space it would be near / depth, and default near is 0.1).
We should probably be using world/view space depth.
Using world/view space depth implies we would be using linear depth.
I always forget what the depth texture is stored as though
NDC, same as frag_coord.z
Can you try plugging in closest_depth instead of d_mm into the disocclusion checks and see if that fixes it?
Sure, I'll play around with it.
There was a problem hiding this comment.
Sounds like we should just use linear depth (so that 0.1 and 0.2 are the same distance apart as 0.9 and 1.0, right?) then. View space vs NDC space doesn't really matter, as either way we just want to check for a ~10% difference.
But you said
I don't think we should use the linear depth here. It will cause TAA to act very differently depending on how close the camera is to things.
which I don't understand why. Did you mean non-linear depth?
There was a problem hiding this comment.
Also, not sure how well we'll actually be able to handle the noisy pixelated plane, that's kinda a torture test. I'm more concerned about regular geometry and materials.
There was a problem hiding this comment.
I don't think we should use linear depth. Yes 0.1 and 0.2 are the same distance apart as 0.9 and 1.0, but depending how close the camera is to something it will dramatically change the distances involved. Things in the far distance will often much more easily have significantly larger distance differences, while appearing close together on screen.
One example of how to think about this it to think about a normal sized scene, say the indoor part of bistro. If you scale the whole scene up 10x and make the camera move 10x faster, every thing will look and feel the same, but the distances will be a lot further and TAA will totally break. On the other hand if you scale everything down to miniature size, make the camera move slower, it will again look the same, but TAA will not disocclude nearly as much as at normal scale.
I think your initial comment about "checking if the difference in depth is greater than ~10%" is probably closer to what we want and would require either not using linear depth or taking into account the pixel radius.
|
In light of #8974 (comment) I think we should hold off on this and discuss our options for 0.12. If this behavior is only present for worst case "super noisy" textures, adding in more complexity here might actually make the common case worse (given that the seams on the FXAA fallback line are pretty stark ... at least for me while in motion). |
|
@DGriffin91 could you open a new PR / modify this one to make FXAA a function in order to allow FXAA to be able to be called from TAA? I'll then integrate it into my PR. |
alice-i-cecile
added
the
X-Controversial
| // Read the color at the new UV coordinates, and use it. | ||
| var finalColor = textureSampleLevel(view_target, linear_sampler, finalUv, 0.0).rgb; | ||
| return vec4<f32>(finalColor, centerSample.a); | ||
| } |
| // Settings for FXAA EXTREME | ||
| const EDGE_THRESHOLD_MIN: f32 = 0.0078; | ||
| const EDGE_THRESHOLD_MAX: f32 = 0.031; |
There was a problem hiding this comment.
Why are these here and not in the FXAA shader modules?
|
I'm abandoning this in favor of using https://github.com/DGriffin91/bevy_mod_taa which addresses many issues with the current TAA implementation. |
Update
I'm abandoning this in favor of using https://github.com/DGriffin91/bevy_mod_taa which addresses many issues with the current TAA implementation.
I also don't think falling back to FXAA is a great approach.
Objective
Solution
This PR also updates the
anti_aliasingexample with a noise texture and camera movement needed to illustrate the issue.These images show the camera under motion. View at full resolution and note right side of image.
Current TAA:
FXAA fallback:
No FXAA fallback:
