[design] alpha channel handling

[aurelienpierre]

So far, modules/nodes are assumed to have opaque alpha channel (== 1 for each pixel) but the pixel data is RGBalpha nonetheless. Alpha blending can happen using a blending node.

The limitations of this design are:

• we should add a blending node for each masked node, which might create a cluttered node graph if a simple opacity is used for blending,
• we always assume that the alpha channel will be defined outside of a node, but for a drawing node, alpha is part of the brushes and will be tempered with inside of the node.

In the spirit of a full-fledged Porter-Duff alpha compositing with minimal pipeline assumptions and general use case, here is what I propose:

1. Each node will have as input :
   • RGBalpha, coming from the previous node, and premultiplied. Let's call that alpha "alpha". It should be forced to 1 (full opacity) in nodes performing raw processing filters and color adaptation.
   • an extra alpha input, taking an user-defined mask (parametric or drawn) for that node. Let's call that "mask alpha".
2. Each node will process either :
   • the alpha-premultiplied RGB internally ? (as agreed on IRC, just force alpha = 1.0 if opacity needed)
   • the alpha-demultiplied RGB (assuming full opacity), then re-apply the alpha ?
   • probably DSP-style and usual raw processing filters will use the latter, but drawing and creative nodes might prefer the former. Maybe this could be user parametric.
   • painting nodes will add new RGBalpha layers that will be merged and flattened with input RGBalpha before the next step.
3. Each node will output :
   • premultiplied RGBapha using both alpha AND mask alpha : { R * a * ma, G * a * ma, B * a * ma, a*ma }, meaning alpha <- alpha * mask alpha.
4. Each node output gets blended, but then I'm lost between possibilities :
   • current darktable has a linear pipeline which always assumes that a module input is an opaque background and blends the module output as a transparent foreground.
   • proposed vkdt has a non-linear pipeline that can use branches, and each branch (if using current proposal) will inherit the transparency of its previous nodes until the branch is explicitly composed (and it has to) against some opaque background at some point, using a blending node.
   • what if "simple" users want the simple blending transparent node output OVER opaque node input ? Do we still force them to use a blending node ? Do we duplicate a simple blending in-node for easiness ?
   • use a generic blending.h API and expose it full in-node. The blending node will be made of only this, other nodes will inherit it too but with background forced to node input and foreground force to node output. (as agreed on IRC)
5. Blending node will have :
   • 2 RGBalpha inputs:
     • background
     • foreground
   • 3 unary controls by input, as defined in Porter-Duff paper [1], used to post-process the background and foreground RGBa in the blending context :
     • darken/brighten d, such that RGBa = { r * d, g * d, b * d, a } (useful to use multiply blending mode in place of overlay, to overlay a texture on a base image. The foreground texture could be adjusted such that "grey" hits 1, then it would act as a bumping map under the multiplication).
     • dissolve e, such that RGBa = { r * e, g * e, d * e, a * e } (dissolve(F, 0.5) PLUS dissolve(B, 0.5) = physically-accurate double exposure).
     • opacity o such that RGBa = { r, g, b, MAX(a * o, 1) } (useful to force an opaque background)
     • the background and foreground unary factors could have a linked mode that ensures their sum{d, e, o}_(back, front} == {1 , 1, 1 }.
   • a blending mode:
     • Porter-Duff classics (OVER == NORMAL, PLUS, IN, OUT, ATOP, XOR),
     • algebraic (MULTIPLY, DIVIDE, SUBTRACT, ADD == PLUS),
     • statistics (GEOMETRIC MEAN, HARMONIC MEAN),
     • and perhaps display-referred bullshit (SOFT/LINEAR/HARD-LIGHT and such).
   • 1 RGBalpha output, perhaps with an opacity override control ?
   • an option to choose between correlated, and uncorellated geometry to properly switch between alpha blending models (or try to implement [2] which appears to provide a more general approach). See [3] to get a sense of the implications.

Tell me what you think.

[1] http://graphics.pixar.com/library/Compositing/paper.pdf
[2] http://graphics.pixar.com/library/DeepCompositing/paper.pdf
[3] https://blender.stackexchange.com/a/67371