Add import/export support for AVIF image format

[cryptomilk]

This adds support to import and export images in the AVIF image format. As the format is really new, it is not widely supported yet. However as most of the big players like (Google, Mozilla, Amazon, Netflix, ...) are investing in AV1. We will see support for it in Android, Firefox, Chrome etc. sooner or
later. exiv2 also doesn't have support for it yet.

To fully support this, we need to also solve #3719

More details are at:
https://blog.cryptomilk.org/2019/11/26/avif-image-support-for-darktable/

[TurboGit]

I can probably take care of the wide gamut colorspace support. I need to find the proper primaries matrix and transfer functions for each of them. From there is should be easy.

[cryptomilk]

That would be awesome. Maybe @joedrago can help, he might have the primaries and transfer functions already in https://github.com/joedrago/colorist/ which also uses lcms2.

[TurboGit]

Ok, primaries for 2020 we already have. For P3 I have just added them, trivial to find. Now I just need a clear description of what is the PQ and HLG transfer functions.

[cryptomilk]

I think PQ is described here: https://www.itu.int/rec/R-REC-BT.2100-2-201807-I/en

[TurboGit]

Please can you test with #3752.

I have the four new color spaces, but they are giving very bright images, maybe expected when not using a HDR image? Not sure... maybe something wrong in may code.

[cryptomilk]

Awesome thanks, I will rebase this on your branch.

I think it is normal that they are too bright as we don't have the display to show them. A lot of example images are at:

https://github.com/AOMediaCodec/av1-avif/tree/master/testFiles/Netflix/avif

[cryptomilk]

For me, they are very dark, at least the test files which are 10bit rec2020 hlg compared to those which are 8bit bt709 gamma 22.

[TurboGit]

When I said "too light" it was when using the new profiles as "input color profile" for a standard RAW. So probably expected.

You're speaking about the export ? When using one of the new profiles as output profile ?

[TurboGit]

You forced pushed your changes with mine :) When (if) I update my branch you'll have to properly rebase by grafting your changes on top of mine. You can handle this case ?

[cryptomilk]

I'm often handling 100+ patches in a working branch for month or even years :-) I guess I can deal with it. I've pushed an additional patch which also adds the missing profiles names.

[TurboGit]

@cryptomilk : noted, sorry for asking but I had to deal with some mess sometime :)

[TurboGit]

I've seen you message on pixls.us, so my wide-gamut are ok ?

[cryptomilk]

I guess they are correct, but I'm not 100% sure.

[joedrago]

Hi all, I got an email ping from @cryptomilk on this thread, and I just wanted to mention that you can use my colorist tool to make really basic PQ test images to verify your pipeline. For example:

colorist generate "256x256,#ff0000..#000000" test.png

will make a really obvious red-to-black gradient image in rec709 with a 2.2 gamma, which you can then convert (any image) to a 10bpc rec2020 PQ lossless AVIF with:

colorist convert test.png test.avif -b 10 -p bt2020 -g pq -l 10000 -q 100

These pixels should have wildly different codepoints, but should represent the exact same xyY values when converted to darktable's rec2020-linear colorspace. You can also view them in Vantage if you have macOS or Windows handy and drag around with the right mouse button to see how I interpret the values.

I'll attach the files I explained above in case you don't want to bother getting any more tools.

testimages.zip

[cryptomilk]

The AVIF with PQ Rec2020 loads fine. It is completely black, but if I increase the exposure, then I start seeing the gradient. So I think this is fine.

I guess it would be interesting to test darktable on a HDR screen.

[joedrago]

This means you are missing the luminance scaling part. PQ is implicitly a 10,000 nits container, whereas most other transfer functions are relative luminance (zero is black and one is "as bright as the monitor can go"). After darktable converts to linear, it should multiply each channel by (10000/destLum), where the destination luminance is how bright you consider SRGB to be (or how bright you plan to display on the screen). Chrome always uses 80 when displaying PQ on SDR displays, Vantage lets you adjust it with the T keybind. assuming dark table handles overranging correctly (channels>1), everything should look really good.

…

On Sun, Dec 15, 2019, 3:45 AM Andreas Schneider ***@***.***> wrote: The AVIF with PQ Rec2020 loads fine. It is completely black, but if I increase the exposure, then I start seeing the gradient. So I think this is fine. I guess it would be interesting to test darktable on a HDR screen. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3749?email_source=notifications&email_token=AACRENUD4Y3D7CVV3YYFNSDQYYKG5A5CNFSM4J2Z2F72YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEG4XIIY#issuecomment-565802019>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACRENVD3LPCXLDVNEVTCIDQYYKG5ANCNFSM4J2Z2F7Q> .

[TurboGit]

@joedrago : I see, this means that our color spaces are OK but when we are using them for whatever image on SDR display we need to adjust the final image by multiplying the 3 RGB channels by around 80. Is that correct?

[joedrago]

No, you don't multiply by 80, you scale by the ratio of "source luminance" to "destination luminance" when going between overranged linear and PQ. Here's Chrome's implementations of FromLinear and ToLinear, for example:

https://cs.chromium.org/chromium/src/ui/gfx/color_transform.cc?l=97
https://cs.chromium.org/chromium/src/ui/gfx/color_transform.cc?l=177

The big question to ask yourself is "how bright, in nits, is (1.0, 1.0, 1.0, 1.0) when darktable is operating in rec2020 linear?" Chrome has locked in on 80 in their implementation (likely due to SRGB's semi-unused standard), macOS might think it is 100 nits, nominal HLG might prefer it was 203 nits, etc.

Does darktable offer a means to show how bright a pixel is already? If so, what math is it doing to calculate the value in nits? That should be the value you use in a ratio alongside 10000 when going to/from PQ.

[cryptomilk]

@TurboGit so where should do that multiplication for each channel? When reading the AVIF file is probably wrong as we need to know the display profile?

[joedrago]

Scaling the channels like this only makes sense after converting to linear; scaling in PQ (or any other nonlinear TF) will do weird things.

In my opinion it should just be a part of the PQ->Linear and Linear->PQ conversions, assuming darktable's LinearRec2020 has a known luminance for (1.0,1.0,1.0) and does good things with overranging (channels > 1.0).

[TurboGit]

@joedrago : Well sadly this is looking beyond my expertise :( Hopefully someone will shine in and help with the matter as to how best integrate this in dt.

[cryptomilk]

@TurboGit I've rebased this on master.

[TurboGit]

@cryptomilk : is that still ok for merging given that we do not scale the luminance as discussed above? Or is that independent and workable after? Bear with me, I know next to nothing about wide-gamut images...

[cryptomilk]

I've created a bug for the PQ to linear part. I think this is good to go. For the beginning AVIF 8bit is interesting as it gives better quality with reduced file size.

[TurboGit]

Ok, I'll merge on master and we will merge this in 3.0.1 when luminance scaling is done.

Thanks for opening the issue.