Order independent transparency

[ffreyer]

Using commit #dceac53

with

using GLMakie
scene = Scene()
r = Rect3f(Point3f(-1, -0.1, -1), Vec3f(2, 0.2, 2))

ps = [Point3f(0, y, 0) for y in (0, 0.2, 0.4)]
cs = [RGBAf(1, 0, 0, 0.3), RGBAf(0, 1, 0, 0.5), RGBAf(0, 0, 1, 0.7)]

meshscatter!(scene, ps .+ Point3f(0, 0, 0), marker = r, color = cs[[1, 2, 3]], transparency = true)
meshscatter!(scene, ps .+ Point3f(1, 0, 0), marker = r, color = cs[[2, 3, 1]], transparency = true)
meshscatter!(scene, ps .+ Point3f(0, 0, 1), marker = r, color = cs[[3, 1, 2]], transparency = true)
meshscatter!(scene, ps .+ Point3f(1, 0, 1), marker = r, color = cs[[1, 3, 2]], transparency = true)
meshscatter!(scene, ps .+ Point3f(1.1, 0.1, 1.1), marker = r, color = :gray, transparency = false)
cam3d!(scene)
center!(scene)
scene

See #1390

I'll try to summarize my changes a bit:

For OIT I added a RGBA{Float16} color buffer and a N0f8 buffer. The former holds the sum of transparent colors and weights. The latter holds the product of all (1-alpha_i). They are written to in fragment_output.frag via mustache replacements, so transparency is a static per plot just like shading. The summation and the product of colors, weights and alpha values work via blending functions. All transparency = true plots are rendered and then merged into the main color buffer (via OIT postprocessor) at the same stage, after SSAO and before FXAA.

I also made some general adjustments:

• I removed the second framebuffer (which only had the color_luma in it)
• I reduced the position buffer for SSAO to Float16 I went back on this because the positions are not normalized. I'm worried using Float16 might break SSAO in a large coordinate space. I'm not sure if it works in the first place though.
• reduced position buffer from 4 floats to 3
• I'm reusing the float16 colorbuffer from OIT as a normal_occlusion buffer for SSAO and as color_luma for FXAA.
• I added a function to add colorbuffers with automatic id finding. This relies on GL_COLOR_ATTACHMENTs to be successive numbers.
• I adjusted enabletransparency() to keep the output alpha at 1 for the opaque color buffer. I also adjusted the OIT blending shader to ignore other values. One of those two seems to be necessary.

After merging this pr, rendering will require

• 4*8 color + 2*32 objectid + 24+8 depth & stencil + 4*16 transparent color & weight + 8 alpha = 200 Byte per pixel without SSAO active (8 more than before with FXAA active, 72 more with FXAA off)
• 200 + 3*32 position = 296 Bytes per pixel with FXAA+SSAO active (152 less than before)

[ffreyer]

Before I end up forgetting this - I removed the second framebuffer because it doesn't seem necessary to me. Because of that there is only one id here. FXAA now uses a second texture/color buffer on the primary framebuffer instead.

[ffreyer]

I also added a new function to attach color buffers. It looks through the already attached ids and figures out the next one automatically.

[ffreyer]

One problem with OIT is that colors can get overexposed in the transparency pass because its color buffer isn't fixed to a (0, 1) range. The current output for overlapping panes (lit front, unlit back) looks like this:

We could explicitly clamp(color, 0, 1) before applying weights to combat this effect. The result looks like this:

Or a more extreme clamp(color.rgb, 0, color.a):

All the images use panes with colors RGBAf(1, 0, 0, 0.5), RGBAf(0, 1, 0, 0.5) and RGBAf(0, 0, 1, 0.5). I'm not sure if clamping is the right choice here, especially the last case since it may distort colors. But we probably want some kind of color/brightness adjustments here, right?

[ffreyer]

Looks like enable_depth in volume messes with the depth_shift test. Also scenekw = (show_axis = false,)) in LScene doesn't work anymore.

[SimonDanisch]

Yeah the axis moved out of scene into lscene, so you need to directly pass show_axis to the lscene

[ffreyer]

Also is the upload of the test images gone or am I just blind?

[SimonDanisch]

Yeah they changed the UI it seems... The only way I found them was navigating to summary:

And on that page you'll find the artifacts

[ffreyer]

I changed \sum_i alpha_i * w(z_i, alpha_i) from the paper to just \sum_i w(z_i, alpha_i). That fixes the issue of overexposing and it fixes an issue with transparent grays getting mapped to white (i.e. rgb >= 1).

Example with alpha = (1, 0.75, 0.5, 0.25):

Example with mixed alpha (blue/magenta with alpha = 0.25, green/cyan = 0.5, red/yellow = 0.75):

[lazarusA]

this is looking great!

[ffreyer]

The current implementation is reading from and writing to the same color buffer. Is this something I should avoid?

I switched to doing it with blend functions.

[ffreyer]

This breaks contour(rand(10, 10, 10), levels=[0.5]) for some reason...

Fixed it

[ffreyer]

I added a documentation page on transparency. Do I need to include it in some list somewhere or is it automatically included?

[SimonDanisch]

Awesome! :)
It's here:
https://makie.juliaplots.org/previews/PR1418/documentation/transparency/
under documentation!
You want to do the honor of merging?