crash in rav1e.dll

[novomesk]

This issues affects GIMP when saving some (not all) images in AVIF format.
https://gitlab.gnome.org/GNOME/gimp/-/issues/7397

GIMP uses libheif which prefer to use rav1e encoder to compress data with AV1 codec.

The problem could be reproduced also without GIMP, by using heif-enc.exe from mingw64/mingw-w64-x86_64-libheif package.

testfile.zip

$ heif-enc.exe -A -o output.avif Avichan8bit.png
Segmentation fault

Similar command in my Linux doesn't crash (I have rav1e 0.4.1 + libheif 1.12.0).

Thread 1 received signal SIGSEGV, Segmentation fault.
0x00007ffb8c113322 in ?? () from C:\msys64\mingw64\bin\rav1e.dll
(gdb) bt
#0  0x00007ffb8c113322 in ?? () from C:\msys64\mingw64\bin\rav1e.dll
#1  0x00007ffb8c1efc61 in ?? () from C:\msys64\mingw64\bin\rav1e.dll
#2  0x00007ffb9695e125 in ?? () from C:\msys64\mingw64\bin\libheif.dll
#3  0x00007ffb96937120 in ?? () from C:\msys64\mingw64\bin\libheif.dll
#4  0x00007ffb9693761b in ?? () from C:\msys64\mingw64\bin\libheif.dll
#5  0x00007ffb96937e59 in ?? () from C:\msys64\mingw64\bin\libheif.dll
#6  0x00007ffb9692a0bf in ?? () from C:\msys64\mingw64\bin\libheif.dll
#7  0x00007ff68f367080 in ?? ()
#8  0x00007ff68f3613b1 in ?? ()
#9  0x00007ff68f3614e6 in ?? ()
#10 0x00007ffbb5e07034 in KERNEL32!BaseThreadInitThunk ()
   from C:\WINDOWS\System32\kernel32.dll
#11 0x00007ffbb6062651 in ntdll!RtlUserThreadStart ()
   from C:\WINDOWS\SYSTEM32\ntdll.dll
#12 0x0000000000000000 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

I suspect the root of the problem could be somewhere in mingw64/mingw-w64-x86_64-rav1e package. Maybe rebuild with help?

On the other hand, saving AVIF via libavif (which can also optionally use rav1e), works. I tried following command:
avifenc.exe -j 4 -c rav1e -y 420 --min 25 --max 55 --maxalpha 30 -o output2.avif Avichan8bit.png

If it is not possible to resolve the problem quickly, there is possible temporary workaround to build libheif without rav1e (-DWITH_RAV1E=OFF). libheif will use alternative AV encoder libaom and the heif-enc.exe command will work without crashing.

[Jehan]

Note that the popped-up error tells about an "unhandled exception" (cf. this message) so I was wondering if it could be related to the issue from last week: #9771

In other words, maybe just rebuilding whatever broken package (rav1e, libheif?…) now that you have a working gcc could fix it (I'm not sure I got what was the exact problem back a week ago though… Now every unhandled exception looks like it could be related! 😆)?

[novomesk]

Let's close here in MSYS2.

Problem should be addressed in libheif (maybe next version) and in GIMP I implemented workaround to select AOM encoder.

If some other project experience this crash too, libheif package can be rebuilt with -DWITH_RAV1E=OFF to disable using rav1 encoder until a fixed version is available.