Color picker not reporting data for rgb hsl hsv

[todd-prior]

Likely a side effect of the work you are doing and you likely know .... Color picker seems to show the right color selected and represented in the patch and reports values for Lab and LCh but not the other options... Observed after building Win version tonight...

[mediatechnologic]

Thank you Todd, I can confirm this too.
May be Aurelien can fix my reported color picker, display and export issues on this occasion?

Even the export of downscaled JPG or PNG images is not gamma correct.
This is REALLY an issue for all fine details in photos, because they will turn
at least too dark and if it goes badly even with wrong color.

That is one reason, why I am using ImageMagick for sharpening all JPG-exports
gamma-correct specially for each output display (TV, projector, screen, web …)
and each output resolution.
This output sharpening (filtering) is a basic requirement of sampling theorem
and is a function of image resolution, viewing distance, camera ISO and more.

It is really a pleasure to use Ansel.
After only some minutes of "playing" with it, I am impressed by
the significantly clearer interface and concentration on the essentials.
Thank you, thank you, thank you!

[todd-prior]

Thanks for sharing all your info. I have not had time to go over it. I suspect it will make sense if I break it down... I started that one long link you shared but ran out of steam.....

[aurelienpierre]

Even the export of downscaled JPG or PNG images is not gamma correct.
This is REALLY an issue for all fine details in photos, because they will turn
at least too dark and if it goes badly even with wrong color.

What ??? Please open a separate issue for this with more details, raw file, etc.

[aurelienpierre]

Color picker seems to show the right color selected and represented in the patch and reports values for Lab and LCh but not the other options... Observed after building Win version tonight...

Yep, I'm aware of that.

[mediatechnologic]

Even the export of downscaled JPG or PNG images is not gamma correct.
This is REALLY an issue for all fine details in photos, because they will turn
at least too dark and if it goes badly even with wrong color.

What ??? Please open a separate issue for this with more details, raw file, etc.

Yes, in some minutes I will post details with test image and calculation methods
in the DT issue "sRGB export wrong without linear scaling calculation"
after my post here darktable-org#13335 (comment)

[aurelienpierre]

Again, I don't understand.

Interpolation, aka downscaling, is done in linear RGB no matter what, for exports as well as in previews. In the pixel pipeline, it's achieved by the "finalscale" module, which comes before the gamma one.

So I'm not sure what is your "ground truth" here, but maybe that should be checked first.

[todd-prior]

I guess if you look at that checkerboard and see how it gets handled which do you believe to be true. The result offered now or the result you would get if the colors were converted to linear sRGB then averaged and then converted back. In one case the result is rose and the other it's a light green??? This is just for the picker but I think the post is extrapolating this to scaling

[mediatechnologic]

I understand, that not everybody likes synthetic images, because real world is much better. But for checking tools and understanding modules it is not bad. See Boris with his new DT Episode 64 from yesterday.

In fact with DT v4.2.0 the color picker, the previews and the exports are wrong. It is not higly relevant, but somehow not calculated linear. Easy to check and see with a checkerboard image. Make this in Gimp and everything is not only ok but perfect. Even the small previews in the GUI and also the very small previews in the "file open" menu.

I have updated DT export issue darktable-org#13335 (comment) as proposed with an image. Not a raw file, but a tif which is highly suitable and impressive for scaling tests.

[aurelienpierre]

Ok I see what's going on.

The finalscale (final interpolation) module goes before the gamma module, but the gamma module doesn't actually do what it says… It only converts the 32 bits float pipeline to 8 bits int.

The mistake comes from finalscale going after colourout (output color profile), which contains the OETF to display/export space.

It's an easy fix that requires to reorder the pipeline by default, but will need a new version of this default order. That will not fix the color picker, but… anyway, I'm not sure what color picker use is aside from debugging stuff.

On the bright side, full-size exports are unaffected.

[aurelienpierre]

Pending a new pipeline order, this has been fixed like this :

1. The interpolation (finalscale) module is made visible in GUI :
   
2. The label in the module links to this explanation : https://ansel.photos/en/doc/modules/processing-modules/finalscale/
3. User may do the correction themselves if needed.

I have checked the validity of the downscaled image, and the pixel average holds to the first decimal, when sampled in [0; 255] in float. There is still some error depending on the interpolator, but that can't be avoided. Bilinear will need to be used for maximum accuracy, though it makes the picture blurrier.

By the way, here is a quick Python snippet used to test the outputs:

from PIL import Image  # install  : `pip install Pillow`
import numpy as np      #install : `pip install numpy`
import colour                 # install : `pip install colour-science`
 
def avg_channels(file):
    im = Image.open(file)
    pix = np.array(im) / 255.
    pix_linear = colour.sRGB_to_XYZ(pix)
    return pix_linear[...,0].mean() * 255, pix_linear[...,1].mean() * 255, pix_linear[...,2].mean() * 255

# Original picture, 32 Mpx
avg_channels("land_ocean_ice_lights_8192.tif")
# return : (2.9592709002720587, 2.7644535889115707, 5.5187406900319846)

# Downsampled picture, 2 K, bilinear
avg_channels("land_ocean_ice_lights_8192-ds-linear.jpg")
# return : (2.9725819889595715, 2.7864168923077699, 5.5815076350389798)

# Downsampled picture, 2K, bicubic
avg_channels("land_ocean_ice_lights_8192-ds-cubic.jpg")
# return (2.9896959342008254, 2.801286675947789, 5.6262843419955706)

# Downsampled picture, 2K, Lanczos 2
avg_channels("land_ocean_ice_lights_8192-ds-lanczos.jpg")
# return (2.9909853661400159, 2.8029255600263019, 5.6273309820864394)

# Downsampled picture, 2K, bilinear, default darktable 3.0 pipeline order
avg_channels("land_ocean_ice_lights_8192-ds-linear.-1jpg")
# return : (2.5032398286613953, 2.2797819829383346, 5.1432525850279136)

[mediatechnologic]

Thank you very much Aurelien. I like the method to link in the GUI to "Learn more"
and even more your content on it with clear good explanations.

The hint to do now the final/output sharpening after downscaling is also good. Basically in the pipeline from camera sensor to display/print you need physically a lot of filtering and sharpening. Most people do not know the technical reason, if, why and where in the pipeline sharpening does happen or is required.

[aurelienpierre]

Thanks. That's a perk of being the dictator of the doc…

[AlynxZhou]

While the exporting works now, color picker is still failed on RGB or HSV, any idea on where to find the problem?

[AlynxZhou]

I finally squeeze some time to try to fix it by myself: looks like @aurelienpierre previously dropped histogram profile, but forgot that color picker uses the histogram profile converted value, which are all zero now, it can be simply fixed by using display profile picked color.

A PR will be followed soon.

[todd-prior]

I finally squeeze some time to try to fix it by myself: looks like @aurelienpierre previously dropped histogram profile, but forgot that color picker uses the histogram profile converted value, which are all zero now, it can be simply fixed by using display profile picked color.

A PR will be followed soon.

Nice.