Revamp D65 chroma correction workflow #15461

jenshannoschwalm · 2023-10-21T08:20:45Z

With the modern scene-referred workflows we do chroma correction in three parts:

We use the 'camera reference' data found in temperature module assuming they are D65
translate what we have to working profile in 'input color profile'
Do a "refinement step" in color calibration assuming we had D65 in step 1

This leads to improved color data in follwing modules - thus it was introduced.

Unfortunately there are some downsides from this approach.
Some modules want accurate white balance data for best quality, most notably
highlights and raw ca correction.
Both modules modify data found in the rgb channels towards "best guessed white" but can't
do so as those data are currently not available.

This PR implements a
chroma struct in dev (like a proxy) holding all required and available white balance data for
the current image (D65, as_shot and currently used) allowing runtime
a) check - are we using D65?
b) access of modules to the other coeffs for improved results
c) easier and safe checks for chroma correction related trouble messages

EDIT: (25.10) temperature modules gets a button introducing this mode and colorin does the "late correction".

Fixes #14518

Related to #15121 and discussions about highlights & opposed

wpferguson · 2023-10-21T15:41:13Z

Some modules want accurate white balance data for best quality, most notably
highlights and raw ca correction.

And denoise profile.

@rawfiner thoughts?

jenshannoschwalm · 2023-10-21T15:54:30Z

And denoise profile.

@rawfiner thoughts?

Exactly :-) This would be a prerequisite ...

rawfiner · 2023-10-23T06:36:11Z

@wpferguson indeed, denoise profile in Y0U0V0 mode is working better with an accurate white balance

wpferguson · 2023-10-23T18:38:32Z

Just tested. Great improvement on denoise profiled :-D. I have a directory of Play Raws and I('ve gotten 2 crashes trying to import it, so there's something going on. I'll see if I can track it down.

jenshannoschwalm · 2023-10-23T18:41:44Z

Leave this for a day or two. Will need more care.

wpferguson · 2023-10-24T00:49:55Z

It appears to be choking on RAF files. Is there something different about Fuji white balance?

wpferguson · 2023-10-24T18:22:50Z

@jenshannoschwalm, not your bug. See #15477

jenshannoschwalm · 2023-10-24T18:45:57Z

@jenshannoschwalm, not your bug. See #15477

Here is constant work on this so force pushing what is stable working yet.

Next will be proposal for white balance improved.

So far working good with better trouble messages and infrastructure only...

jenshannoschwalm · 2023-10-25T16:24:57Z

Ok, from my side it seems to be working fine all over, would love

testing this new workflow @rawfiner maybe
feedback on design flaws - (anything just plain wrong?)

There are now 4 commits

1&2 introduce the new chroma struct, some refactoring and make sure the module trouble message work fine. So something i suggest to get reviewed @TurboGit

3&4 introduce the simplistic user interface, temperature module version required bumping (required both for user interface and for cache safety without playing with iop cache data). Implementation in colorin needs more love for sure, will follow soon if tests go fine.

wpferguson · 2023-10-25T16:59:38Z

Reading the log messages I found

All chroma related warnings will be served via the colorbalancergb module instances.

Did you mean color calibration module instances?

I'm testing now. Do you have any particular test cases you want me to try?

jenshannoschwalm · 2023-10-25T17:06:52Z

Did you mean color calibration module instances?

YES FOR SURE! channelmixerrgb or color calibration, just fixed the commit message :-) THANKS!

jenshannoschwalm · 2023-10-25T17:13:06Z

Do you have any particular test cases you want me to try?

Nope. Whatever and However your workflow is. Modules resulting in a visible difference will mostly be those mentioned above.

TurboGit · 2023-10-25T17:14:44Z

YES FOR SURE! channelmixerrgb or color calibration, just fixed the commit message :-) THANKS!

I was commenting about this too...

TurboGit

First review on the code.

src/develop/develop.h

src/iop/temperature.c

TurboGit · 2023-10-25T17:08:09Z

src/iop/temperature.c

@@ -1622,6 +1661,7 @@ void reload_defaults(dt_iop_module_t *module)
    dt_bauhaus_combobox_add(g->presets, C_("white balance", "user modified"));
    // old "camera neutral", reason: better matches intent
    dt_bauhaus_combobox_add(g->presets, C_("white balance", "camera reference"));
+    dt_bauhaus_combobox_add(g->presets, C_("white balance", "late camera reference"));


We need more comment about this as I'm not sure what it represents exactly and will have trouble during translation to French. Maybe a better wording for this new option is possible?

The tooltip added below is:
set white balance to as shot and later correct to camera reference point
But then what is the difference between camera reference and late camera reference?

I agree and the wording is just "preliminary". I am not even sure we want the extra button at all. From my understanding - and if all reviewers and testers agree on this - i would

keep old behaviour for current image developments

use new algo by selecting the reset to default

use new algo for new edits.

But all are open for discussion.

src/iop/temperature.c

wpferguson · 2023-10-25T18:14:20Z

Tested highlight reconstruction, raw ca and denoise profiled.

Highllight reconstruction worked without issue in all modes. It seemed to me that I might have got a little more detail in the recovered highlights, but it could be just my imagination.

raw ca worked well. I tested against the roof image in the demosiacing improvement issue, as well as a couple of others. raw ca either cleaned up all of the ca or most of it. For the most of it instances applying the ca module finished it up.

denoise profiled was night and day for me. I shoot lots of high ISO images (sports, bad lights, high shutter speeds). denoise profiled in master leaves white "noise" artifacts in the dark areas and doesn't always do a really good job of denoising the lighter areas, because it doesn't have access to the white balance coefficients. So after denoising, I apply an instance of D or S with fine denoise to get rid of the artifacts, and then maybe a second instance (either denoise medium or denoise coarse) to smooth out some of the luma noise. This process yields a fairly good result and keeps the room warm :). With this PR the dark areas have no white artifacts, and the lighter areas have some luma noise on really high ISO images. One instance of D or S cleans up the luma noise if I need to. The resulting image is noticeably better than what I can currently achieve from master.

TurboGit · 2023-10-25T18:29:12Z

denoise profiled was night and day for me. I shoot lots of high ISO images (sports, bad lights, high shutter speeds).

I tested on some high iso pictures and I hardly see any difference. How did you test?

jenshannoschwalm · 2023-10-25T18:47:03Z

From theory in highlights the blown out parts should be corrected more to white instead of yellowish. For raw ca correction the New stuff seems almost always better.

jenshannoschwalm · 2023-10-25T20:18:44Z

One more experience, also the LMMSE algo works slightly better.

todd-prior · 2023-10-25T21:51:28Z

profiled in master leaves white "noise" artifacts in the dark area

While unrelated to this PR wrt your comment...the default slider for preserve shadows I have found leads to a lot of that... simply dropping that back in a measured way really helps.... likely you have explored this but if not then you could check...

wpferguson · 2023-10-25T23:35:55Z

I tested on some high iso pictures and I hardly see any difference. How did you test?

It appears that I tested apples against oranges. I used a high ISO image that I had previously developed and compared the noise reduction of that to the PR. I failed to take into account the other steps that I used developing the image in the first place which had an effect on the output. Once I developed them the same and the only difference was the noise reduction, the difference was much closer.

I am glad you asked the question and made me go back and look my method because in that testing I found a nasty little bug unrelated to this PR. Issue coming soon.

Since introducing the modern chroma correction workflow we do the chroma correction in two phases. First we use the daylight/D65 coeffs in temperature Second we refine for better data in colorbalancergb. Unfortunately 1) some modules would prefer better white-balanced data, notably highlights, raw chromatic aberration as their algorithms assume white-is-white / resp all rgb signals are equal for greytones. 2) due to the implementation in temperature we don't have all coeffs (currently used, D65 and as-shot) publicly available but only internal in gui data. 3) warnings are difficult to handle This pr implements and uses a struct `dt_dev_chroma_t` in `dt_develop_t` holding all relevant data. The temperature module reads them all and we can check at all stages of the pipe via dt_dev_check_d65(), also we avoid keeping coeffs in temperature gui data. Some refactoring and renaming for readability

All chroma related warnings will be served via the channelmixerrgb (color calibration) module instances. The one being `chroma.adaptation` also knows about the temperature module status, so it writes the trouble status there too. For increased responsivness and situations without UI updates (initial loading in darkroom) we update the troubles after the preview pipe has finished.

We now have a second bulb button on the very right introducing the "late D65" mode. The effect will be: 1. the temperature module will process data with "as-shot" coeffs and keeps track on this in the dev chroma struct. 2. also the processed_max and dsc data are using this 3. See colorin module how this get's resolved Otherwise the algos have not changed. Presets are supported as before

Preliminary code for evaluation of the algo 1. only cpu code so far 2. some performance penalty

Donatzsky · 2023-10-26T11:00:27Z

How does this interact with custom WB coefficients?
I use custom WB coefficients as explained by AP in this video, to correct the yellow cast of my camera. Will this invalidate that in some way?

I'm on stable, so can't test it, but I can share a raw and my coefficients.

jenshannoschwalm · 2023-10-26T15:31:53Z

Will this invalidate that in some way?

Short answer is no, i suppose you mean having presets for temperature.

This pr changes the pixelpipe color maths only for the new "D65-late" button. All modules until colorin will use the as-shot coeffs (those are often - not always - better than the known D65 coeffs) and are finally fixed to D65 in the input stage of colorin. So no interference for other workflows.

Donatzsky · 2023-10-26T20:03:11Z

Short answer is no, i suppose you mean having presets for temperature.

Not sure we are talking about the same thing, although I do use a preset. What I've done is create custom D65 coefficients by taking a photo of a calibrated white screen, and used that to get what you see in the attached screenshot. Without, my photos have a pronounced yellow tone, but with these coefficients color calibration CAT is pretty much spot-on.

This pr changes the pixelpipe color maths only for the new "D65-late" button. All modules until colorin will use the as-shot coeffs (those are often - not always - better than the known D65 coeffs) and are finally fixed to D65 in the input stage of colorin. So no interference for other workflows.

So, if I understand correctly, there will be a new "D65-late" option (in addition to the current four) in white balance that should usually work better than "camera reference". And if the user chooses any of the others, then everything works as now.

jenshannoschwalm · 2023-10-28T05:46:42Z

#15505 has replaced this so closing

jenshannoschwalm force-pushed the temp_and_chroma branch 3 times, most recently from ec4684d to 27cbe27 Compare October 23, 2023 04:51

jenshannoschwalm marked this pull request as draft October 23, 2023 17:25

jenshannoschwalm force-pushed the temp_and_chroma branch from 27cbe27 to a329052 Compare October 23, 2023 18:21

jenshannoschwalm force-pushed the temp_and_chroma branch from a329052 to e6bc729 Compare October 24, 2023 17:41

jenshannoschwalm force-pushed the temp_and_chroma branch 2 times, most recently from 7fd0433 to ef3eb85 Compare October 25, 2023 16:13

jenshannoschwalm marked this pull request as ready for review October 25, 2023 16:15

jenshannoschwalm added feature: enhancement current features to improve feature: new new features to add scope: image processing correcting pixels scope: color management ensuring consistency of colour adaptation through display/output profiles release notes: pending labels Oct 25, 2023

jenshannoschwalm changed the title ~~Revamp chroma correction workflow (I)~~ Revamp D65 chroma correction workflow Oct 25, 2023

jenshannoschwalm force-pushed the temp_and_chroma branch from ef3eb85 to 4b61b10 Compare October 25, 2023 17:09

TurboGit requested changes Oct 25, 2023

View reviewed changes

jenshannoschwalm force-pushed the temp_and_chroma branch 2 times, most recently from 45eca0d to 5e77ade Compare October 25, 2023 17:53

jenshannoschwalm added 4 commits October 26, 2023 07:06

Make colorin do the late D65 correction

5f06467

Preliminary code for evaluation of the algo 1. only cpu code so far 2. some performance penalty

jenshannoschwalm force-pushed the temp_and_chroma branch from 5e77ade to 5f06467 Compare October 26, 2023 05:07

jenshannoschwalm mentioned this pull request Oct 26, 2023

Rework temperature and channelmixerrgb #15497

Merged

jenshannoschwalm mentioned this pull request Oct 28, 2023

Introduce as-shot to later D65 workflow #15505

Closed

jenshannoschwalm closed this Oct 28, 2023

jenshannoschwalm deleted the temp_and_chroma branch October 28, 2023 05:53

TurboGit removed the release notes: pending label Nov 6, 2023

jenshannoschwalm mentioned this pull request Nov 8, 2023

Introduce as-shot to later D65 workflow (improved highlights) #15602

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Revamp D65 chroma correction workflow #15461

Revamp D65 chroma correction workflow #15461

jenshannoschwalm commented Oct 21, 2023 •

edited

wpferguson commented Oct 21, 2023

jenshannoschwalm commented Oct 21, 2023

rawfiner commented Oct 23, 2023

wpferguson commented Oct 23, 2023

jenshannoschwalm commented Oct 23, 2023

wpferguson commented Oct 24, 2023

wpferguson commented Oct 24, 2023

jenshannoschwalm commented Oct 24, 2023

jenshannoschwalm commented Oct 25, 2023

wpferguson commented Oct 25, 2023

jenshannoschwalm commented Oct 25, 2023 •

edited

jenshannoschwalm commented Oct 25, 2023

TurboGit commented Oct 25, 2023

TurboGit left a comment

TurboGit Oct 25, 2023

TurboGit Oct 25, 2023

jenshannoschwalm Oct 25, 2023

wpferguson commented Oct 25, 2023

TurboGit commented Oct 25, 2023

jenshannoschwalm commented Oct 25, 2023

jenshannoschwalm commented Oct 25, 2023

todd-prior commented Oct 25, 2023

wpferguson commented Oct 25, 2023

Donatzsky commented Oct 26, 2023

jenshannoschwalm commented Oct 26, 2023

Donatzsky commented Oct 26, 2023

jenshannoschwalm commented Oct 28, 2023

Revamp D65 chroma correction workflow #15461

Revamp D65 chroma correction workflow #15461

Conversation

jenshannoschwalm commented Oct 21, 2023 • edited

wpferguson commented Oct 21, 2023

jenshannoschwalm commented Oct 21, 2023

rawfiner commented Oct 23, 2023

wpferguson commented Oct 23, 2023

jenshannoschwalm commented Oct 23, 2023

wpferguson commented Oct 24, 2023

wpferguson commented Oct 24, 2023

jenshannoschwalm commented Oct 24, 2023

jenshannoschwalm commented Oct 25, 2023

wpferguson commented Oct 25, 2023

jenshannoschwalm commented Oct 25, 2023 • edited

jenshannoschwalm commented Oct 25, 2023

TurboGit commented Oct 25, 2023

TurboGit left a comment

Choose a reason for hiding this comment

TurboGit Oct 25, 2023

Choose a reason for hiding this comment

TurboGit Oct 25, 2023

Choose a reason for hiding this comment

jenshannoschwalm Oct 25, 2023

Choose a reason for hiding this comment

wpferguson commented Oct 25, 2023

TurboGit commented Oct 25, 2023

jenshannoschwalm commented Oct 25, 2023

jenshannoschwalm commented Oct 25, 2023

todd-prior commented Oct 25, 2023

wpferguson commented Oct 25, 2023

Donatzsky commented Oct 26, 2023

jenshannoschwalm commented Oct 26, 2023

Donatzsky commented Oct 26, 2023

jenshannoschwalm commented Oct 28, 2023

jenshannoschwalm commented Oct 21, 2023 •

edited

jenshannoschwalm commented Oct 25, 2023 •

edited