Posterization / Precision Loss with Working Profile Custom Tone Response Curve

[jedypod]

I have searched through the open issues and don't see this mentioned so I am making a new issue to report this.

I am working on a python wrapper for converting camera raw images to scene-referred exr. One of the debayer engines supported is RawTherapee.

In order to use RawTherapee for demoisaic, I need to output linear gamma. The only way I have found to do this is to set Tone response curve: Custom and set Gamma and Slope to 1.0.

Unfortunately it seems that as soon as I switch Tone response curve from None to Custom, there is a huge loss of precision occurring in the image processing.

Here are a couple of example screenshots:

First, with Tone Response Curve set to None:

And here with Tone Response Curve set to Custom:

The profile is exposing up the image so that you can see the posterization in the shadows more easily.

I was thinking of working around this issue by trying to find the equation used for Gamma and Slope... Maybe I could make a lut for the default 2.4 gamma 12.92 slope when Tone Response Curve = None... But I have not been able to locate where this is in the source code. And it seems like a pretty big workaround to an issue that should not be an issue in the first place.

Here is the raw image and the pp3 file used in the screenshot above, attached as a zip file. To reproduce:

• load the image
• go to the Color Management section
• toggle Tone response curve from None to Custom.

M01-1141.zip

[Thanatomanic]

Thanks for reporting, but this tool was completely revamped last year. Can you test the development branch to see if you can reproduce it there?

[Lawrence37]

I can reproduce this in 5.8 but not with a recent dev version.

[TechXavAL]

AFAIK you don't need to tweak the Working Profile TRC at all:

• input profiles usually have a linear TRC, so you only have to choose a good profile for your camera
• as explained in RawPedia, the Working Profile has by definition a linear TRC, so choose the one you like the most, or leave ProPhoto set (as it is by default)
• choose one of Elle Stone ICC linear TRC profiles as the Output Profile and you're done (linear profiles are those finished in -g10)

Don't trust what you see on-screen, as this is the linear TRC image that comes from the engine, but converted to whatever color space and TRC your display uses.

[jedypod]

@Thanatomanic I did attempt to test this compiling directly from the code in the latest commit of the github repo, but trying to open that compilation here on Linux Centos 8 segfaults on start... Do you have any pointers on if there is a specific branch or tag that would be a more stable place to compile from, but with the improvements you speak of?

Thanks very much for the info and links @TechXavAL. Today I learned about the ICC Profile Creator and how to customize the ICC Color Profiles Directory. You are indeed correct that if you customize the icc profiles and create an output profile that has a linear gamma it works. It's too bad that there is not an included output profile that is just a bypass so that it outputs the working profile.

Here's what I did, in case there are other readers with the same problem.

1. Click on the little triangle with the plus button to open the ICC Profile Creator
   
2. Create an ICC Profile with the desired primaries whitepoint and linear tone response curve.
   
3. Save this profile in some folder (in this case ~/.config/RawTherapee/profiles )
4. Click the Preferences button (the icon on the lower left with the audio faders). Customize the ICC Profiles directory in the Color Management preferences to point to the folder you saved the profile.
   
5. Restart RawTherapee
6. Select the ICC profile you created and use it for export.

Thanks again for the help.

[Lawrence37]

The last commit is supposed to be stable. You can try the AppImage from the automated builds. It's a bit old, but I think it has the new version (called "Abstract Profile"). There's a newer experimental AppImage here.

The crash on startup is concerning. If you have the time, please consider reporting the issue along with a backtrace by following the instructions from RawPedia.

[Floessie]

We also have a detailed RawPedia page called "How to Fix Crashes on Startup"...

[jedypod]

@Floessie

We also have a detailed RawPedia page called "How to Fix Crashes on Startup"...

Thanks! I learned some things about how the config and cache directories are set up reading through this page. Unfortunately none of the things mentioned there fixed the crash on start.

@Lawrence37 Thank you for the info! I tried the appimage from the automated builds and it works fine here and the segfault on startup issue is not present.

I did some more digging, compiled with debug, and did a gdb stacktrace (for the first time) following the instructions here.

gdb ./rawtherapee
(gdb) r
Starting program: /opt/compile/rawtherapee/rawtherapee-rpmbuild/BUILDROOT/rawtherapee-5.9-1.el8.x86_64/opt/rawtherapee-5.9/rawtherapee                                                                                                                        
Missing separate debuginfos, use: yum debuginfo-install glibc-2.28-151.el8.x86_64
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()

(gdb) info threads
  Id   Target Id                                         Frame
* 1    Thread 0x7ffff7fa5e40 (LWP 2146886) "rawtherapee" 0x0000000000000000 in ?? ()

(gdb) thread apply all bt full
Thread 1 (Thread 0x7ffff7fa5e40 (LWP 2146886)):
#0  0x0000000000000000 in  ()
#1  0x00007ffff1e1126f in _GLOBAL__sub_I_lens.cpp () at /lib64/liblensfun.so.2
#2  0x00007ffff7ddf8ba in call_init.part () at /lib64/ld-linux-x86-64.so.2
#3  0x00007ffff7ddf9ba in _dl_init () at /lib64/ld-linux-x86-64.so.2
#4  0x00007ffff7dd0fda in _dl_start_user () at /lib64/ld-linux-x86-64.so.2
#5  0x0000000000000001 in  ()
#6  0x00007fffffffcfe2 in  ()
#7  0x0000000000000000 in  ()
(gdb) 

I don't know much about gdb or debugging or doing stack traces, but it looks like there is an issue with the version of lensfun I am linking against. I compiled lensfun 0.3.95 alpha from the master branch of their github page. Maybe their latest is incompatible and I need to compile an older version or the last stable release (0.3.2 ?).

[Thanatomanic]

Maybe their latest is incompatible and I need to compile an older version or the last stable release (0.3.2 ?).

Yes, this could certainly explain it.

[jedypod]

Hey experts, I'm back again.

I realized after more testing that my custom icc profile workaround above has a fatal flaw: it does not work on the commandline.

If I use a pp3 file pointing to a custom ICC profile name, rawtherapee-cli does not use it. I have tried multiple icc profile locations including

• /usr/share/rawtherapee/iccprofiles/
• /usr/share/color/icc/
• custom directory specified in the rawtherapee settings as the icc profile directory
  The output from rawtherapee-cli seems to just bypass the custom icc profile or use a default setting.

Is there any workaround to get this working?

Thanks for your help!

EDIT: I figured this out with a fresh brain! I realized I had put the icc profile in /usr/share/rawtherapee/iccprofiles, and it needed to be in /usr/share/rawtherapee/iccprofiles/output! Woops 😅 move along nothing to see here...