Choice of display profile affects histogram, color picker values and overexposure indicators

[flannelhead]

Describe the bug
It doesn't seem possible to saturate the black level to yield zero RGB values.

black_level_test_file_and_xmp.zip

To Reproduce
Steps to reproduce the behavior:

1. Open a raw image with no edits (an example Olympus ORF file attached)
2. Enable the Exposure module
3. Set black level correction to 1.0
4. Enable under/overexposure indicator
5. Observe there are no parts of the image indicated as underexposed (I have the underexposure threshold set at 2% which is the default)
6. Color picker also shows a minimum value of RGB = (6, 6, 6), at some point this becomes independent of the black level correction (e.g. the same result with 0.1 and 1.0)
7. Tried also doing some other adjustments (e.g. contrast, brightness) but no adjustments seemed to reduce the RGB values
8. Increasing the underexposure threshold to 3%, now most of the image is shown as underexposed

Expected behavior
Most of the image should be shown as underexposed and there should be completely black pixels (RGB = (0, 0, 0))

Platform (please complete the following information):

• OS: Arch Linux x86_64
• Version a82f213

Additional context
In my usual workflow I have used the over/underexposure indicators to observe clipping e.g. when doing adjustments in Filmic, but now the underexposure indication seems to be broken. I can try to find the most recent version where this worked (next week) if it is beneficial. This also doesn't seem to depend on whether OpenCL is enabled or not.

[junkyardsparkle]

Can't reproduce; I get plenty of black clipping showing up. Can you reproduce this on a fresh profile with no settings changed?

[flannelhead]

Indeed it reproduces if I remove $HOME/.config/darktable and $HOME/.cache/darktable and the relevant XMP files and start fresh.

It seems to also reproduce on earlier revisions where I clearly remember the clipping indicator working. Kind of a mystery, guess I'll have to investigate if something is broken on my system (package upgrades etc.)

[junkyardsparkle]

Weird. Maybe something to do with color management or a weird display profile?

[flannelhead]

Indeed it seems to be display profile related; if I start darktable when my calibrated display profile is loaded, the issue reproduces. When using the stock display profile, the minimum value I get is now RGB = (5, 5, 5) instead of (6, 6, 6) (still the nonzero result here puzzles me somewhat). This is enough to fire the clipping indicator. Can anyone provide insight on the nonzero values?

Weird that the issue wasn't there earlier - I'm fairly sure the calibrated profile was loaded also before. colord hasn't received updates recently either. Some KDE Plasma packages have been updated though, I'll continue investigating.

[flannelhead]

Ok, so if I set the display profile to sRGB in darktable (instead of system display profile), RGB = (0, 0, 0) is reached. Guess it's good time for me to recap on Linux color management and check if my calibrated color profile is actually ok.

[flannelhead]

I'm indeed getting some pretty weird effects in darktable. I don't think my display profile is at fault.

The first weirdness is that "system display profile" is listed twice in the "histogram profile" menu (right-click on the soft-proof or gamut check buttons).

The other one is related to this piece in the release notes for 3.0.0rc0:

A new profile ‘histogram profile’ has been added on the same pop-up that the softproof one on the darkroom. It controls the color space of the histogram, color picker and overexposed check. When gamut or softproof checks are active the histogram and color picker use the softproof profile, otherwise they use the histogram profile. The overexposed check always use the histogram profile.

So as far as I understand, the selection of "display profile" or "softproof profile" shouldn't affect the histogram, over/underexposure checks or the color picker values in any way as the "histogram profile" setting is used for those (see the image of the relevant menu).

However, what I'm seeing here is that if I e.g. set "histogram profile" to sRGB and then change "display profile" between "system display profile" and "sRGB", I see an evident change in the histogram, color picker values as well as the indicated over/underexposed regions. :(

Edited the title accordingly.

[junkyardsparkle]

I honestly don't know if that's the expected behavior or not... maybe @edgardoh can clarify how this should work.

[parafin]

"system display profile" is indeed listed twice for some reason. But I can't reproduce the change in histogram when changing display profile. When soft-proofing or gamut check is enabled changing soft-proof profile changes the histogram, but display profile still have to effect.

[TurboGit]

@parafin : looking at the code "system display profile" is displayed twice when there is two displays. probably something to fix to be clearer.

[parafin]

I tested on the system with only one display... Where is this code located?

[flannelhead]

As far as I can tell, the profiles are initially loaded here:
https://github.com/darktable-org/darktable/blob/master/src/common/colorspaces.c#L1253

There both display_profile and display2_profile get assigned a category_pos number which makes them appear in the histogram profile menu due to this code:
https://github.com/darktable-org/darktable/blob/master/src/views/darkroom.c#L2048

I don't know the code very thoroughly, however it seems to me that display2 in darkroom.c at least refers to the "preview display profile", not a second display per se.
https://github.com/darktable-org/darktable/blob/master/src/views/darkroom.c#L1986

Therefore it seems a bit illogical to have both display_profile and display2_profile added in the histogram profile menu (notice that none of the other profile menus have "system display profile" listed twice).

About the change of histogram and pixel values when changing the display profile: I can upload my profile later today for examination. The change is not as clear with all profiles (this could be especially the case if the display profile is very close to sRGB).

[parafin]

OK, I see now about 2 displays. I think they need to be distinguishable from each other here.

[flannelhead]

HP E242 #2 2019-09-18 18-43 D6500 2.2 F-S XYZLUT+MTX.zip
Here's the color profile I'm using. To use it in darktable, copy it to $HOME/.config/darktable/color/out (on Linux at least). Switching the display profile between this and sRGB in darktable should show a shift in the "dark" (low-EV) side of the histogram.