Crash when attempting to open NEF files

[e14tech]

Here is the gdb log info:
log.txt

I installed RawTherapee from the package repo for FreeBSD.
Here is the version info:

Version: 5.9
Branch: releases
Commit: 9b85839
Commit date: 2022-11-27
Compiler: cc 14.0.5
Processor: generic x86
System: Linux
Bit depth: 64 bits
Gtkmm: V3.24.2
Lensfun: V0.3.95.0
Build type: Release
Build flags: -O2 -pipe -I/usr/local/include -fPIC -flto=thin -O3 -funroll-loops -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -isystem /usr/local/include -std=c++11 -mtune=generic -Werror=unused-label -Werror=delete-incomplete -fno-math-errno -Wno-attributes -Wall -Wuninitialized -Wcast-qual -Wno-deprecated-declarations -Wno-unused-result -Wunused-macros -fopenmp=libomp -Werror=unknown-pragmas -O2 -pipe -I/usr/local/include -fPIC -flto=thin -O3 -funroll-loops -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -isystem /usr/local/include -DNDEBUG -ftree-vectorize
Link flags: -Wl,--as-needed -lpthread -flto=thin -O3 -funroll-loops -lm -lomp -fstack-protector-strong -L/usr/local/lib -mtune=generic
OpenMP support: true
MMAP support: ON
Build OS:
Build date: UTC
Build epoch:
Build UUID:

I'm running FreeBSD 13.2-RELEASE-p4.

To produce this issue:

• Open RawTherapee.
• Go to a folder that is storing NEF files.
• When viewing the NEF files in the file viewer, double click any NEF file to Open.
• Crash.

Here is an NEF file:
https://drive.proton.me/urls/0891WA1DXC#jgR1f0bve4Uf

Here is the associated PP3 file:
https://drive.proton.me/urls/S8D38725KG#WyNFxfGHq1oZ

[Lawrence37]

Cannot reproduce on a self-compiled RawTherapee both on 5.9 and dev. Unfortunately, the GDB report does not contain enough information because it is from a release build. If you are able to get your hands on a debug build and reproduce the crash, that would provide useful hints. You can also try the AppImage from the download page. If that works fine, the crash could be caused by something in the FreeBSD build.

[e14tech]

Yeah, AppImages are a Linux specific thing.
I'm betting it's FreeBSD specific as it occurs on multiple fresh FreeBSD installs across different computers.

[martin-0]

I was able to reproduce this on both 13.2 and 14.0 FreeBSD. I found out that it works if I start this in GNOME session (gdm->gnome). If I use something else (such as openbox started by startx directly) it crashes. It seems to be related to ICC color profiles.

Crash occurs here:

#0 0x0000000000b9c55f in rtengine::Color::rgb2lab01 (profile=..., profileW=..., r=1, g=1, b=0, LAB_l=@0x8221a4c88: 3.30228268e-17, LAB_a=@0x8221a4c90: -8.34992299e-20, LAB_b=@0x8221a4c8c: 1.12103877e-44, workingSpace=<optimized out>) at /usr/ports/graphics/rawtherapee/work/rawtherapee-5.9/rtengine/color.cc:501

The code snippet from rtengine/color.cc:501

const TMatrix wprof = rtengine::ICCStore::getInstance()->workingSpaceMatrix(profileCalc);

const float xyz_rgb[3][3] = { {static_cast<float>(wprof[0][0]), static_cast<float>(wprof[0][1]), static_cast<float>(wprof[0][2])},
                                  {static_cast<float>(wprof[1][0]), static_cast<float>(wprof[1][1]), static_cast<float>(wprof[1][2])},
                                  {static_cast<float>(wprof[2][0]), static_cast<float>(wprof[2][1]), static_cast<float>(wprof[2][2])}
                                };

The wprof is NULL when not ran from GNOME session.
Full backtrace
backtrace.txt

When I modified the snippet above to see the values:

        printf("%s: trying to get workingSpaceMatrix() using %s \n", __func__, std::string(profileCalc));

    const TMatrix wprof = rtengine::ICCStore::getInstance()->workingSpaceMatrix(profileCalc);

        printf("DEBUG: %s: wprof matrix: %p\n", __func__, wprof);

Working example:

rgb2lab01: trying to get workingSpaceMatrix() using sRGB
DEBUG: rgb2lab01: wprof matrix: 0x11f8880

Failing example

rgb2lab01: trying to get workingSpaceMatrix() using sRGB
DEBUG: rgb2lab01: wprof matrix: 0

[martin-0]

I did some testing - it seems these ICC color profiles are somehow set by GDM (I have no idea, till today I never heard about it).
If I do this:

• start gnome directly from startx - same issue
• start openbox from gdm - works just fine

I still think it's a problem that code assumes wprof will never be NULL but this way segfault can be avoided.

[Lawrence37]

Hi @martin-0, that reminds me of another issue reported some time ago. If the crash experienced by @e14tech is the one you're seeing, it's the same as issue #6357.

[martin-0]

Hi @Lawrence37,

Yes, it seems the same issue.

I played around with this a bit more. The problematic function: https://github.com/Beep6581/RawTherapee/blob/dev/rtengine/iccstore.cc#L505

Both in success and failure scenarios these ICC profiles are loaded.
I modified this function to give me more info (this patch is defaulting to "BestRGB", explanation follows): p1.txt

In the failing situation I found out that "sRGB" profile is the last profile in the map. Debug output of such scenario:

getInstance: ICCStore instance: 0x1443d10
rgb2lab01: trying to get workingSpaceMatrix() usingsRGB
getInstance: ICCStore instance: 0x1443d10
workingSpaceMatrix: wMatrices size: 10, trying to find 'sRGB' in wMatrices; list of profiles:
ACESp0:0x11f98e0
ACESp1:0x11f9880
Adobe RGB:0x11f9b80
BestRGB:0x11f99a0
Beta RGB:0x11f9a00
BruceRGB:0x11f9a60
ProPhoto:0x11f9b20
Rec2020:0x11f9940
WideGamut:0x11f9ac0
sRGB:0x11f9be0
workingSpaceMatrix: didn't find sRGB, defaulting to sRGB
DEBUG: rgb2lab01: wprof matrix: 0x0

In this list you can see that profiles are there. But it just cannot find the last profile for some reason. In my patch I changed the "default" profile to "BestRGB" and it started working.

It was working under gdm because "sRGB" was not the last profile loaded for some reason (WideGamut was).

I'm not that familiar with C++, I don't do much in it. I tried to few things but nothing helped. I don't understand why it cannot find it. Checking memory location with gdb seems ok - expected values are there.

[martin-0]

I was in the middle of my comment when you wrote yours. I did want to send it, it if anything because I invested some time debugging it.

The bug you linked - 6357 - and the PR it refers to is a solution to the problem.
It explains why it is failing on FreeBSD. In my case I used default locales that FreeBSD uses after installation.

I don't do C++ much so I can't comment further. It seems locales are really pain. I guess it's just worth noting that the function expects to find something blindly and uses it further without any sanity check on the results. Especially if that function is locales sensitive.

[Lawrence37]

Those are interesting results. It does look like Glib::ustring is being problematic again.

[martin-0]

There is one thing I don't understand though. In my failing scenario ( no gdm, just startx starting gnome-session) it fails with the output I shared above. In my patch I changed the default profile to something else that is not the last element in map and it was working.

If this is a locale issue why it is failing on last element of map only? I would expect it to fail the same way on the key "BestRGB" the same way it failed on "sRGB".

But it really is working with the C.UTF-8 locale set as mentioned in PR. Maybe some sort of C++ weirdness? I dunno..

[Lawrence37]

It's probably because of the way Glib::ustring represents strings in different locales. "sRGB" may have a different representation while the others are consistent across locales.

[Lawrence37]

@martin-0 I created pull request #6889 which should fix the crash. @e14tech I'm not sure if the crash you encountered is the same as the one brought up by @martin-0. If you are not able to get a GDB report from a debug build or try the fix, you can try changing the locale like @martin-0 did (set the environment variable LANG to C.UTF-8) and see if RawTherapee still crashes or not.

[e14tech]

I added "LC_ALL=C.UTF-8" and it works now!

[Lawrence37]

Great! That means there's a high chance it's the same kind of crash. I'll close this issue and track future updates in #6357.