Material Unification for Rasterization and Ray Tracing And Visbuffer Rendering

[CodingDaniel1]

Hi guys, lately I was thinking about Material Unification Usage in bevy.

Since ray tracing is more and more popular these days, and its pipeline works a bit different compared to rasterization. I wonder is it the right time to introduce material indices to every single material shader variant (aka. a raster pipeline).

This way we have a lot of new possibilities, e.g. bindless, we can use BDA and store each material related uniform data inside their own buffers, and use BDA to store it inside a global material ptr buffer.
And we can introduce visibility rendering for deferred rendering just like how Doom The Dark Ages did it.

More importantly, by giving material indices, ray tracing pipeline is a viable option now. The only difficulty is the shader binding table needs to be rebuilt when needed.

The reason i wanna bring this up is because while implementing my own renderer using this approach but a lot more limited in the current stage. I had encountered the issue with deferred material pipeline. And in the end i settle on the visibility rendering approach, which requires material indices.
Then i did a many_cube (1M cube scattered around) test in both my renderer and bevy. My visbuffer approach consistently outperforms bevy's forward rendering and deferred rendering.

Here is the my own internal renderer gpu performance, the rasterization took 3.6ms in total.

Here is the same scene but in bevy, the rasteization and shading took 8.6ms (forward rendering, i tested with deferred as well, and the performance is similar to forward with no significant improvement or regression)

So i think from the performance benefits, visbuffer rendering should be considered the default deferred rendering approach. And its much better to handle different materials (See DOOM TDA visbuffer talk ).

I would like to work on this for bevy if people think this is the direction to go!

At last, I wanna showcase some footages about the performance in actions. (I have no idea why bevy has stuttering issues when i turn the camera, possibly related to frustum culling and causing the rasterization to stutter?)

bevy
my visbuffer approach

[CodingDaniel1]

The testing GPU i used is a RTX 3070, and the resolution i used for testing is maximized window on a 4K monitor

[JMS55]

Copying this from Discord, my long term plan is to build a much higher level Material API that specifically abstracts away where mesh/material/etc data comes from, which then allows us to adapt it to any rendering method (along with many other improvements).

The raw AI-written dump of my thoughts is here https://gist.github.com/JMS55/a1834d8da2557bac9eda0a105cfe2d82, although I don't suggest anyone actually read it.