Blackmagic Pocket Cinema Camera 4k DNG support

[Generic-Patriot]

Hello, I noticed that the BMPCC4k files do not seem to be properly supported in Rawtherapee. When lossy compression is used, there are weird dark purple pixels at or near the highlights, and full clipped white frame results in purple, as seen in the attatched pictures.

Also, sometimes I see black edges on certain sides on certain resolutions when I set the border to zero.

Link to all 24 possible raw file permutation, from resolution (4k DCI 4096x2160, 4k UHD 3840x2160, 1080p 1920x1080), compression level (Uncompressed, Lossless Compressed, 3:1 Lossy compression, 4:1 lossy compression) , and normal exposed or full white clipped.
https://drive.google.com/drive/folders/1HaqJBSw01AAexOA8Rkkc7ycnmu4D8kRD?usp=sharing

Camera firmware 6.1, and Rawtherapee Dev 5.8
Version: 5.8-2178-g6cc9537ab
Branch: dev
Commit: 6cc9537
Commit date: 2020-06-19
Compiler: gcc\ 10.1.0
Processor: generic x86
System: Windows
Bit depth: 64 bits
Gtkmm: V3.24.2
Lensfun: V0.3.2.0
Build type: Release
Build flags: -mwin32\ -m64\ -mthreads\ -msse2\ -std=c++11\ -mtune=generic\ -Werror=unused-label\ -Werror=delete-incomplete\ -fno-math-errno\ -Wall\ -Wuninitialized\ -Wcast-qual\ -Wno-deprecated-declarations\ -Wno-unused-result\ -Wunused-macros\ -fopenmp\ -Werror=unknown-pragmas\ -Wno-aggressive-loop-optimizations\ -DNDEBUG\ -O3\ -ftree-vectorize
Link flags: -m64\ -mthreads\ -static-libgcc\ -mtune=generic\ -s\ -O3
OpenMP support: ON
MMAP support: ON
Build OS: Linux 4.4.0-101-generic x86_64
Build date: Fri, 19 Jun 2020 21:07:32 +0000 UTC
Build epoch: 1592600852
Build UUID: f41dc068-e4b0-4ab4-90ef-71237ada3513

[Thanatomanic]

Many thanks for the images. I have downloaded them in case anybody in the future needs them again.
Let's see how well we can improve support for these files.

[Thanatomanic]

As a naive first try, I did this

diff --git a/rtengine/camconst.json b/rtengine/camconst.json
index 850cc287c..533b00010 100644
--- a/rtengine/camconst.json
+++ b/rtengine/camconst.json
@@ -2865,6 +2865,11 @@ Camera constants:
         "make_model": [ "samsung SM-N960F" ],
         "dcraw_matrix": [ 6632, -622, -1104, -5659, 14819, 773, -1846, 4465, 4967 ] // ColorMatrix1 from Adobe DNG Converter 11.1
     },
+    
+    {
+        "make_model": [ "Blackmagic Pocket Cinema Camera 4K" ],
+        "ranges": { "black": 0, "white": 65472 }
+    },
 
     // Dummy test entry to test the parser and show the format with all entries active
     {

But then I end up with this (4k_UHD_4-1):

And other files can still show artifacts at the position of the lamp, even though the value of 65472 is definitely the maximum raw value in the file.

[Waveluke]

This may or may not be useful, but I also discovered that Adobe DNG coverter v. 12.4.0.547 gets an “unable to parse file” error when converting the lossy compressed files, though fine with the uncompressed and lossless compressed BMPCC4k DNG files. Updating to v 13.1.0.658, I still had the same issue.

[Waveluke]

I just uploaded samples exposed daylight landscape for the BMPCC 4k to the Raw PXLS sample database, so that there are better samples to work with. All full resolution, one of each compression level supported.

[Entropy512]

Per the discussion over at pixls, PR #6448 appears to fix this issue. I initially saw some issues remaining, but it appears that was due to cached previews/thumbnails from when I looked at the files without the patch. After clearing cache everything looks good.

[heckflosse]

@Entropy512 I think you linked the wrong issue

[Entropy512]

Yup, typoed a 5 instead of a 6