filmicrgb: fix norm handling in v6

[flannelhead]

We need to clamp the RGB norm before calculating ratios. Otherwise things like this can happen:

1. Consider the max RGB norm and input pixels (1, 1, 1) and (1, 1, 2). Filmic white is set such that the source white is at (1, 1, 1).
2. Norms are calculated and they are 1 and 2, respectively.
3. Ratios are calculated: (1, 1, 1), (0.5, 0.5, 1).
4. When the filmic tone curve is applied to the norm, it is also clamped to the [0, 1] range at some point. Now both norms have value 1.
5. Ratios are restored; now the output pixels read (1, 1, 1) and (0.5, 0.5, 1). This is bad as the original blue pixel was brighter than the white!
6. Gamut mapping will preserve the color of the blue pixel and it results in a nasty-looking dark patch in the middle of a bright surrounding (think of e.g. lights where the center is clipped in raw).

To remedy this, we already clamp the norm before calculating the ratios. This way when the ratios are restored, they will retain their brightness ordering. Gamut mapping takes care of finally clamping the brightest highlights back into the target range. I don't know if dumb clamping like this is the best solution, but the filmic curve is not really well defined outside the designed dynamic range i.e. it doesn't extend above the source white point value. Correct me if I'm wrong, please. From brief testing I find the white point adjustment in filmic behaving now much more like I would expect it to - the brightest parts will eventually turn white when bringing down the white point slider. Without this PR, this is not always the case.

This behaviour exists also with the earlier filmic v5 revision but probably the mandatory desaturation has masked this issue.

[flannelhead]

This should fix issues like the one reported in #10976 (comment)

[aurelienpierre]

Correct me if I'm wrong, please.

You are not.

Good job !

[flannelhead]

I think this is correct as-is but something could still be done to make the transitions a bit smoother. I'm thinking about it.

[TurboGit]

Looks good, thanks!

[piratenpanda]

did not change the behaviour as described in #10976 (comment)

[flannelhead]

@piratenpanda thanks for the report. Do you happen to have a raw file + XMP you could share where this behaviour is seen?

[piratenpanda]

Ok might as well be user error. If I lower the white value I can recreate the v5 look now. Sorry for the noise. That's most likely expected behaviour that the white points are different for v5 and v6.