Real-time denoising

[jaxry]

Currently, our renderer draws a scene progressively. The more frames rendered from a given camera angle, the higher quality the resulting render becomes. While progressive rendering is typical for path tracing, it is far from ideal with our goal to bring path tracing into real-time.

Lots of research has gone into methods for denoising a path traced image, with the objective of producing a high quality result in only one frame. I am aware of two papers describing similar methods that would be most applicable to our renderer. The design involves hybrid rendering - the scene is rasterized to several G-buffers, and a 1spp path traced image is spatially and temporally filtered with help from the G-buffers.

• Schied et al. - Spatiotemporal Variance-Guided Filtering
  Shied's algorithm splits the direct illumination and the indirect illumination into two buffers which are then filtered separately. The spatial filter is a edge-avoiding wavelet filter.
• Mara et al. - An Efficient Denoising Algorithm for Global Illumination
  Mara's algorithm keeps direct and indirect illumination together, but instead splits diffuse and specular reflections into their own buffers. These buffers are then filtered separately, and the spacial filter is a bilateral filter.

I'm not sure which approach is better suited for our needs, and I will try implementing both to compare the two. Does anyone have any experience with these two methods? Are there other methods?

[sweco-sekrsv]

I saw these denoising shaders in shadertoy, might be interesting to have alook at:
https://www.shadertoy.com/view/ldKBzG
https://www.shadertoy.com/view/ldKBRV

Not sure if this has denoising but it converges really quick.
https://www.shadertoy.com/view/MsXfz4

[jaxry]

Thanks for the links! That denoising shader seems to implement a cut-down version of SVGF, with its use of a wavelet filter. It may serve as a useful reference when we work on implementing SVGF in full.

[ManishJu]

This beats nvidia Iray in many ways:
http://www.cs.tut.fi/~foi/papers/Koskela-TOG-2019-Blockwise_Multi_Order_Feature_Regression_for_Real_Time_Path_Tracing_Reconstruction.pdf
Their source code is also available

[jaxry]

This looks like an excellent paper! It reports a ~2x speedup over SVGF, and the spatial filter looks conceptually simpler to implement as well. I've been starting to explore this technique, and I hope to see some results in a few weeks.

Does Nvidia Iray use the OptiX AI Denoiser that is compared to in the paper?

[ManishJu]

Yes! Iray uses the Optix Denoiser.

[RenderTool]

Here is a denoise blog, I hope it will be useful to you.
https://alain.xyz/blog/raytracing-denoising

[barnabasbartha]

Can somebody write a quick update about how the development of the real-time path tracing is going? I'm really interested and super excited to see the outcome or the challenges this issue is facing with.

[jaxry]

Hi @BarthaBRW . The biggest WIP feature right now is on this branch https://github.com/hoverinc/ray-tracing-renderer/tree/demodulate-albedo

The branch separates diffuse and specular light into separate buffers, and demodulates surface textures so that they're first applied in the post-processing step. It'll bring some big visual quality improvements because it'll increase the effectiveness of the temporal filter. But more importantly, it lays the necessary groundwork for the spatial filter.

The specifics of how to implement the spatial filter is up in the air though. I believe using BMFR (posted above) for diffuse light, and using a modified SVGF for specular light would give us the best possible results, but it's hard to say before trying it. Might you have any ideas?

[barnabasbartha]

Thanks @jaxry for the quick response. I'll need some time to catch up some of these topics but looking forward to contributing.

[jaxry]

The pipeline described in NVIDIA's Minecraft RTX implementation looks like it fits the requirements of our renderer too!

https://www.youtube.com/watch?v=mDlmQYHApBU

They use SVGF on a diffuse buffer and a specular buffer separately, and a special anisotropic kernel is used for the sky shadows. Light in their implementation works very similar to what we have, so their solutions likely would work great.

[jaxry]

Just giving a heads up: I'm spending all my time working on a full denoising pipeline as we speak, and I hope to get some good results in a few weeks. Expect other development on the renderer (like better Three.js support) to be on the backlog for now.

[trsh]

Is it ready? :)