Segfault when opening compressed CR3 and cropped raw from Canon R5

[Thanatomanic]

See here for the original report: https://discuss.pixls.us/t/canon-r5-cropped-raws/25441

This file does not decode, but instead segfaults R5__0596.ZIP. It is shot as a lossy compressed cRAW in APS-C mode with a crop factor applied. ExifTool tells me:

Compression=JPEG (old-style)
ImageWidth=ExifImageWidth=CroppedImageWidth=SourceImageWidth=5088
ImageHeight=ExifImageHeight=CroppedImageHeight=SourceImageWidth=3392
AspectRatio=3:2 (APS-C crop)
SensorWidth=5248
SensorHeight=3510
SensorLeftBorder=144
SensorTopBorder=108
SensorRightBorder=5231
SensorBottomBorder=3499

There are two issues at play.

• The support for cRAW may be fixed by looking at https://www.libraw.org/news/libraw-202101-snapshot.
• The support for the crop factor can only be investigated after that

In any case, it shouldn't segfault.

[heckflosse]

Here's a quick and very dirty patch to avoid the segfault. It's not meant to be committed, but as a starting point for a better patch.

diff --git a/rtengine/dcraw.cc b/rtengine/dcraw.cc
index ef0b4e8dc..e631d4828 100644
--- a/rtengine/dcraw.cc
+++ b/rtengine/dcraw.cc
@@ -4397,6 +4397,11 @@ void CLASS crop_masked_pixels()
       }
     }
   } else {
+    // dirty hack to show what's wrong and avoid segfault
+    if (height == 3510 && width == 5248 && !strncmp(model, "EOS R5", 6)) {
+        top_margin = 0;
+        left_margin = 0;
+    }
 #ifdef _OPENMP
 #pragma omp parallel for
 #endif

Maybe another dev ( @Thanatomanic ) can make a more correct patch.

Ingo

[Thanatomanic]

The main issue is that our camconst.json format can only handle one particular raw crop per camera, whereas this camera (and others like it) can have multiple depending on a shooting mode. The smaller raw image sizes are read correctly by dcraw, but the crop values in camconst indicate a bigger file, so too much memory is being read leading to the segfault. The actual segfault happens in this function

RawTherapee/rtengine/dcraw.cc

Line 4371 in 166538d

void CLASS crop_masked_pixels()

I don't think we should be casually hacking that. A slightly better workaround may be similar to this #6255

To be continued...

[Thanatomanic]

For reference, @agriggio has implemented raw size-dependent crops: https://bitbucket.org/agriggio/art/commits/c8df72cfd2b0c15fda1f5d5c3c5364161daff1bc

[Floessie]

I like how Alberto adopted my style for writing lambdas. 😎

[virxkane]

I guess same problem with CR3 raw files from Canon EOS R and 1.6 crop mode.
log.txt
current git 'dev' branch

[EGD]

Same problem with Canon EOS R6 (#6330).

[heckflosse]

I did some hacking. Here's a first result. Still needs some work.