RAWs from Canon 7d have pinkish highlights

[maboleth]

I imported some of my old Canon 7D RAW files into DT and upon opening them in darkroom all of them had pink highlights that I simply couldn't turn off.

The first line is how the images should look like, the 2nd row is after opening them in DT.

Normally, I've seen this behaviour after messing with filmic rgb and/or highlight recovery. Or stretching out whitebalance.
However, here are out of the box settings - white balance set to "as shot", no HL reconstruct, no filmic rgb.

In fact, HL recon. set to "reconstruct in LCh" does seem to ease this color off, but it's still not entirely.

Latest DT master git, arch linux.

[junkyardsparkle]

It's completely normal for blown highlights to be magenta when the highlight reconstruction is turned off, that's why it's on by default. :-)

After hitting the reset button on that module (which should set "clip highlights" mode with threshold of 1) is the color cast still there?

[maboleth]

After hitting the reset button on that module (which should set "clip highlights" mode with threshold of 1) is the color cast still there?

Yes, it's going from purple (hl recon. off) to pink (hl recon. on). The only thing that removes it more or less successfully is lowering the threshold to 0.65 or so. However, that's also clipping some light highlight details in the sky.

[junkyardsparkle]

Hmm... a couple of random thoughts: I believe there's an "enhanced" input profile for that camera? If it's selected in the input color profile module, does changing to "standard matrix" make any difference? If the base curve module is enabled, does changing the chroma preservation mode make any difference? Also, I notice the difference in the thumbs you posted seems to correlate to ISO setting... does this only happen with certain ISO values?

[maboleth]

Hmm... a couple of random thoughts: I believe there's an "enhanced" input profile for that camera? If it's selected in the input color profile module, does changing to "standard matrix" make any difference?

That's correct and that doesn't make any difference in this case.

If the base curve module is enabled, does changing the chroma preservation mode make any difference?

That actually did it! Though not chroma preservation but turning it to 'none' altogether. That did the trick. However, on really pinkish/purple images it still displayed slight pink hues. Like 1%.

Also, I notice the difference in the thumbs you posted seems to correlate to ISO setting... does this only happen with certain ISO values?

It is a bit. ISO 320 is more pronounced than ISO 250. I noticed a similarity though with different model, but still Canon. Interestingly, Canon 5d MKIV drastically changes RAW b/w values in DT from ISO 250 to ISO 320. It's only 1/3 step difference, but it is a night and day in module's values.

Overall, can this whole issue be somehow subdued or explained, why does it happen and what us, users can do when it happens (curves, hl. reconstruct etc.)?

On a more advanced level - can it automatically be corrected by DT in the future?

[junkyardsparkle]

Though not chroma preservation but turning it to 'none' altogether. That did the trick.

Well, normally the default "luminance" mode should be pretty safe even with blown highlights, so if you're still seeing some magenta cast even with "none" then something is off somewhere else... maybe the RAW b/w points need re-evaluating? I don't have much experience there, never really needed to worry about them for my cameras. :-)

[maboleth]

Well, normally the default "luminance" mode should be pretty safe even with blown highlights, so if you're still seeing some magenta cast even with "none" then something is off somewhere else... maybe the RAW b/w points need re-evaluating? I don't have much experience there, never really needed to worry about them for my cameras. :-)

What worries me in this case(s), I cannot have base curve disabled. Simply put, if I import my image with only highlight reconstruction and white balance enabled (excluding always-on modules like demosaic, raw bw point etc), the images come out with horrible pink highlights. So horrible that they look like this:

And its histogram:

This is the imported Raw file with reset raw bw module to default, white balance set to 'as shot' and highlight recovery to 'clip highlights'. Nothing else.
Also I cannot have linear base curve - I need to have it curved (brightened the image) to push those pink hues out. So effectively, base curve isn't remedying this by itself, it's literally pushing the pink out of the highlight histogram values when the line is heavily curved.

Same is true to any exposure-altering module - simply making the exposure overexposed or bright, will make pink disappear by pushing it out.

[jenshannoschwalm]

You should really share the raw file and sidecar for anyone to analyse

[maboleth]

Alright, thought about it but wanted to be sure if there's something to be done on my end.

Here it is. You don't need a xmp file, just import it and see in the darkroom.

_MG_4771.zip

[RawConvert]

@maboleth I'm wondering if dt has the wrong white values for your camera. If you go to the "raw black/white point" module and reduce the white to around 13,000 (from default 16,300 on my system) the pink/magenta disappears. This white point setting depends on ISO usually - certainly with my eos 6D - and dt should set it appropriately as long as your camera is supported.

[jenshannoschwalm]

ok, the red channel is vastly out of bounds due to overexposure. As you said you can set the clipping threshold in highlight reconstruction to something like 0.6. The white balance shows a very strong red channel too so the "overblowing" is so heavy.

It could be also a problem of the rawspeed decoder getting wrong white points ...

[jenshannoschwalm]

maybe @LebedevRI knows

[wpferguson]

Here's a couple of xmp files on how to deal with it. If you switch on the raw over exposure you see the red and blue channels are both blow out.

4771.xmps.zip

[RawConvert]

@maboleth , I haven't looked at the XMPs above, however, to add to what I said previously, I remembered there is a parameter file used by Rawtherapee to derive the white point, camconst.json. I've changed the name to .txt in order to upload it here. This has values for both 7D models. For the ISO 250 you were using, the white point RT uses is 15,200, not the 16,300 my dt had.
camconst.txt

[parafin]

It does look like incorrect white level. Probably these intermediate non-real ISO values are affected. @LebedevRI should take a look.

[LebedevRI]

Yep, that's pretty typical indication of wrong white level.
As it can be seen, for that camera, there is no specific white level specified for ISO250:
https://github.com/darktable-org/rawspeed/blob/develop/data/cameras.xml#L897

The simple solution is for someone who owns that camera to take one shot
(even body cap on) per every ISO level that could possibly be selected on that camera,
and i'll adjust the values.

The better solution is to fix RawSpeed to read those whitelevels from CR2 automatically.

In both cases, the existing developed images will regress, much like in the case with changing color matrix.