Color balance module : add the ASC CDL mode

[aurelienpierre]

As Blender and many video editors, add a color balance mode based on the ASC color decision list https://en.wikipedia.org/wiki/Color_Decision_List

This allows to color-grade in linear ProPhoto RGB space, whereas the previous version was in gamma-corrected sRGB and led to saturation issues.

The previous mode is retained as an option. The legacy params are imported. The OpenCL and SSE2 versions are provided and working (as far as I tested).

[aurelienpierre]

Warning : with DNG files, changing the main power value gives either a green or a magenta color cast. Tested with NEF, RAF, ARW and CR2 files, it works as expected.

[aurelienpierre]

DNG files work now as expected. I have added the fulcrum contrast and master saturation options.

In the pure C variant, just enabling the module gives an unexpected exposure boost. Something is wrong with the Lab <-> ProphotoRGB conversion in C, I can't find what. (I have checked the conversion matrices from 3 sources). Using sRGB fixes the problem for the pure C version (I suspect the gamma conversion does the trick).

SSE and OpenCL versions behave as expected with Prophoto RGB conversion and everything.

I have done a bit of lib refactoring. I think rewriting the SSE and OpenCL color conversions as native dot products would give a boost in performance at some point.

[TurboGit]

Good job Aurélien. I'm probably not the right person to merge this PR, but it looks good to me and works as expected. I expect houz or hanatos to have a deeper look. Thanks.

[aurelienpierre]

Thanks Pascal ! If you compile my master branch, you will get it along with the Log unbreak profile, and both together work really well to recover high dynamic range images.

[aurelienpierre]

I have added HSL sliders in addition to the RGB ones, as I requested here https://redmine.darktable.org/issues/11959.

This doesn't change the data structure (parameters are still RGB). Editing RGB sliders updates HSL ones, and the other way around.

I still have a problem of slight magenta shift with the SSE ProphotoRGB conversion, OpenCL and pure C behave as expected.

I have rewritten SSE color conversion function in algebraic notation instead of SSE intrinsics to improve readability.

[edgardoh]

I've playing with this as well, very nice! Results are very good and is very easy to get them.
Maybe add a combo with lift/gamma/gain/(all) so it doesn't take so much space on the UI?
Also, is it possible to have a new group for the entire range (not only shadows/highlights/midtones)? Sometimes I like to create a color cast and then play with each range in particular.

[aurelienpierre]

Maybe add a combo with lift/gamma/gain/(all) so it doesn't take so much space on the UI?

You mean on the text label ?

Also, is it possible to have a new group for the entire range (not only shadows/highlights/midtones)?

Like a master hue setting ?

[edgardoh]

Maybe add a combo with lift/gamma/gain/(all) so it doesn't take so much space on the UI? You mean on the text label ?

I mean to add a combo box with those entries that show/hide each group (or all of them if (all) is selected)

Also, is it possible to have a new group for the entire range (not only shadows/highlights/midtones)? Like a master hue setting ?

Exactly. It can be added to the combo box above, maybe as a default.

[aurelienpierre]

I mean to add a combo box with those entries that show/hide each group (or all of them if (all) is selected)

I still don't understand 😁 You want to selectively enable/disable the RGB set of sliders and the HSL ones ?

Exactly. It can be added to the combo box above, maybe as a default.

The problem with that is it would change the maths behind and lose the compatibility with the ACL Color Decision List standard. Technically-speaking, the slope is sort of the master (it's the linear factor), and the first parameter you should set in your workflow. Then, you can come back a bit on it by setting the offset, which, again applies to the whole range but, numerically, weighs more on the low-lights. And as the last step, you fix the power. That's why the mode is called slope/offset/power, in the order you are supposed to adjust the settings.

[edgardoh]

I mean to add a combo box with those entries that show/hide each group (or all of them if (all) is selected) I still don't understand 😁 You want to selectively enable/disable the RGB set of sliders and the HSL ones ?

Not disable but hide it, kind of what the channel mixer does, but with the option of displaying them all. But if you're not going to add the master hue, I can live with all the sliders visible, they fit in my screen anyway.

…

[aurelienpierre]

I have added neutralization color-pickers on each hue slider. The idea is to select a color that should be neutral (grey) and compute the right parameters in the color balance to invert its current hue and neutralize it. The saturation is not always on point though. It's still useful to remove indesirable color casts, due to several lighting sources at different color temperature.

[aurelienpierre]

Another improvement here : I have bounded the RGB sliders and corrected them according to the luminance.

Now, when you edit a RGB channel, the 2 others are updated so that the luminance is not affected. So, the RGB channels control only the saturation and hue, making them fully independant from the factor (which is a luma correction).

I think that's pretty cool.

[TurboGit]

You need to remove the rawspeed submodule from here.
Also there is a small conflict to be resolved.

I think also that it should be better to rebase all this and maybe even squash all the commits together at this stage. No need to see all the "fix" steps now.

[aurelienpierre]

Isn't rawspeed listed in the gitignore that it keeps showing up in the commits ?

I will fix that, maybe add color-pickers for lift and gain factors, and I think we wil be good.

[TurboGit]

Ok, let me know when ready. We are approaching the code freeze I suppose. So if we want this in the next release (and I think it would be really nice) I need to review and merge soon. Thanks.

[aurelienpierre]

I think it's clean ?

So, last changes:

1. I added a load of comments to explain the maths happening in the optimizers
2. I added the optimizer for luma distribution, as promised
3. these optimizers should be a tad more robust in difficult cases (I added constraints)
4. I removed the RGB sliders bound (for luma normalization) for the legacy lift/gamma/gain, so people can continue to use it as before. It's still there for the
5. I cherry-picked the common lib colorspaces_inline_conversions.h that I optimized, corrected, and made more readable by replacing the SS2 special functions by algebraic operations (following the advice of one Rawtherapee dev, because it seems algebraic operations can be more optimized by the compilator).
6. I added the SIMD() parameter for openmp loops
7. I added some more info to the labels.

Screenshots:

HSL sliders:

RGB sliders:

Both sliders:

Directions to test:

• Always set-up the slope, then lift, then gamma in this order.
• Always set-up the luma (factors) before the color (RGB/hue/saturation).
• The factor color-pickers take the max luma in the zone for the slope, the min luma in the zone for the offset, the average luma for the power. Choose your zones accordingly. Once you have set 3 samples (slope/offset/power), you can trigger the "optimize luma" optimizer. It will fit the histogram inside the whole dynamic range [0-100 %] and normalize the average luma to 50 %, similarly to the levels but in RGB.
• The hue color-pickers aim at reverting the color of the samples you give them to neutralize color casts. They take the average color of the zone. Once you have set the 3 samples (slope/offset/gamma), you can trigger the "neutralize color" optimizer. The samples should have a neutral color (grey-ish) in real life in low/mid/high-lights (if you take real saturated colors, I don't garantee the result). That's usefull when you have several white balances due to several light sources in the same picture.
• Set the master saturation and contrast in last
• The grey fulcrum is the luma value that is non-affected by the contrast adjustement. In portrait, use the color-picker to select the whole face and get its average value. Then, the contrast you will add will enhance the shape and features. 18 % Luma = 50 % L(ab) so you should always get 18 % if the gamma/power factor was set properly.

Note that the color-pickers are designed for the slope/offset/power version, as well as the HSL -> RGB conversion and the RGB luma normalization. But they still work fairly for lift/gamma/gain.

You will notice that if you try RGB -> HSL -> RGB' conversions on the sliders, for primaries, you will always have an error : RGB = RGB' ± 0.005. That's expected, due to the conversion of color shiftts between a cylindric and a cartesian vectorial space. However, ping me if you get errors >= 0.01, it shouldn't happen.

Enjoy !

[TurboGit]

One issue found:

• open module, select "both" for "color control sliders"
• click on the color picker IN the color picker module on the left
• the color balance module is reset to "HSL" mode only instead of both

[TurboGit]

Another one:

• open a new picture
• activate color balance
• let all default value
• click on the picker for the factor in offset
• draw an area
  => nothing happens

Is that expected? Maybe a wrong procedure on my side?

[aurelienpierre]

The first issue is a real issue. For the second, you need to click once on the color picker, draw the area, then click again on the color picker icon to validate the color.

[TurboGit]

Seems like the crash is fixed. Cannot reproduce, same on your side now?

[aurelienpierre]

It depends how I build. With cmake, it seems to work. With build.sh, it fails sometimes.

[aurelienpierre]

still outputs, when closing :

Thread 1 "darktable" received signal SIGSEGV, Segmentation fault.
(gdb) bt full
#0  0x00007ffff3e8234c in g_module_close () at /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0
#1  0x00007ffff7a7e1af in dt_view_unload_module (view=0x555557817470)
    at /home/aurelien/Documents/Programmes/darktable/src/views/view.c:228
        iter = 0x555557772580
#2  0x00007ffff7a7e1af in dt_view_manager_cleanup (vm=<optimized out>)
    at /home/aurelien/Documents/Programmes/darktable/src/views/view.c:101
        iter = 0x555557772580
#3  0x00007ffff796be61 in dt_cleanup ()
    at /home/aurelien/Documents/Programmes/darktable/src/common/darktable.c:1066
        init_gui = 1
#4  0x00007ffff7a55b41 in dt_gui_gtk_run (gui=<optimized out>)
    at /home/aurelien/Documents/Programmes/darktable/src/gui/gtk.c:1201
        widget = <optimized out>
        allocation = {x = 362, y = 12, width = 401, height = 601}
#5  0x0000555555554830 in main (argc=<optimized out>, argv=<optimized out>)
    at /home/aurelien/Documents/Programmes/darktable/src/main.c:83

[TurboGit]

And related to OpenCL, I cannot reproduce with standard code path.

[aurelienpierre]

So it was because I had

codepaths/openmp_simd=true
codepaths/sse2=true

in darktablerc. Seems to work now.

[TurboGit]

And when I replace colorbalance_cdl() implementation with a simple call to colorbalance (in extended.cl) I don't have the crash (the image is not correct of course). So it looks like the issue is in the CL code in colorbalance_cdl(). Now I'm far from expert in OpenCL :(

[aurelienpierre]

I don't understand : what crash do you get with colorbalance_cdl() OpenCL ? Both versions work with no error on my side, with the last commit.

[TurboGit]

When I'm leaving dt I get segmentation violation ini different part of the application (not always the same). This seems to indicate that there is some memory corruption somewhere.

[aurelienpierre]

@TurboGit I have commented out the faulty lines that store the current color into the private data structure, like that:

#ifdef OPTIM
    dt_iop_colorbalance_data_t *d = (dt_iop_colorbalance_data_t *)self->data;
    d->luma_patches[GAIN] = XYZ[1];
    d->luma_patches_flags[GAIN] = 1;
#endif

Can you confirm everything works that way so that we ensure these are the true offenders ?

[aurelienpierre]

I think I have found it.

The private data structure (dt_iop_colorbalance_data_t *) is stored in (dt_dev_pixelpipe_t *)pipe->data. Here, I'm writing in (dt_iop_module_t *)self->data which is something else, hence the memory corruption.

But thanks to the barely-commented and non-documented sourcecode of dt, I have not found how to access the pixelpipe from the module structure.

[TurboGit]

Ok, no more crashes!

And indeed valgrind (I run a session last night, very very slow under valgrind!) told me that d->luma_patches_flags[GAIN] = 1; was writing 4 bytes out of bound!

So you've found the offenders :)

[TurboGit]

No more crashes but I cannot run the optimizer.

[aurelienpierre]

So you've found the offenders :)

That only took 18h 🙄

No more crashes but I cannot run the optimizer.

that's normal, it's disabled for debugging. I still need to figure out how to reach the pipe data from the module.

[aurelienpierre]

Ok, now I store the data in dt_iop_colorbalance_gui_data_t so it should be clean !!! 😄

[TurboGit]

Yes, all good now! Thanks.

[edgardoh]

May I point that this can break backwards compatibility? https://github.com/darktable-org/darktable/pull/1734/files#diff-66d4f98e149ad915383f3c1cd14d7a46R213 El mar., 16 oct. 2018 a las 10:45, Pascal Obry (<notifications@github.com>) escribió:

…

Merged #1734 <#1734> into master. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1734 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/APq1pT90fhLTUPbWT9MvH1tFqK6GLhEiks5uleL0gaJpZM4W1fED> .