Dolby Vision with wrong colors

[Doofussy2]

mpv version and platform

Windows 10

mpv 0.31.0-11-g49cbc5017c Copyright © 2000-2019 mpv/MPlayer/mplayer2 projects
  built on Sun Dec 29 09:31:44 +08 2019
  ffmpeg library versions:
  libavutil       56.38.100
  libavcodec      58.65.100
  libavformat     58.35.101
  libswscale      5.6.100
  libavfilter     7.69.101
  libswresample   3.6.100
  ffmpeg version: git-2019-12-28-6399eed4

Reproduction steps

Play some Dolby Vision videos. Not all videos play this way, and I tested with and without an mpv.conf. Same result.

Expected behavior

Play with correct color

Actual behavior

Picture plays in purple and green

Log file

Portable mpv log.txt

Sample files

Can be found, here

[Akemi]

reminds me a bit of #4340, though it isn't the same issue.

[haasn]

There's little we can do here because Dolby Vision is a closed spec.

There's a branch that sort of generates the approximate result, but not for all files, and not quite correctly. The specifics of the encoding depends on proprietary metadata which we haven't managed to decode / reverse engineer yet.

[Doofussy2]

Ah, ok so it's going to be hit and miss whether a video will play, correctly. Am I to understand that you guys are actually working on it?

[haasn]

It would be hit or miss on whether a video will play "vaguely watcheable but obviously wrong" or "completely and utterly wrong".

I'm not actively working on it, no. Unless any progress is made on reverse engineering the proprietary dolby vision metadata. If you want to help you could gather sample clips that fall into different "categories" (not all profile 5 media has the same kind of color shift) and seeing what parts of the dolby black box metadata changes in response. That might help pinpointing where in the metadata the color space info is.

[Doofussy2]

Well, the first comparative examples I can give, are:

Correct color

https://www.demolandia.net/downloads.html?id=1039472921

Incorrect color

https://www.demolandia.net/downloads.html?id=543675873

I don't know if that is useful, but there does appear to be a disparity in the HDR format. Note the BL+RPU and BL+EL+RPU. I have no idea what difference the 'EL' makes.

Of the samples I have, the files that have incorrect color, don't have that 'EL'.

[Doofussy2]

OK, I grabbed more DV samples. The pattern, holds. All of the videos that don't have that 'EL', don't play correctly.

I just wanted to say: FUCK Dolby and their racketeering-like patent trolling. Fuck them for re-introducing proprietary bullshit into modern video technology too. I wish they'd fucking die already.

[Doofussy2]

Well yeah, there's that lol. I kind of agree. Proprietary 'secret sauce' stuff is a bit nonsensical.

[Doofussy2]

Stream dumping video from here, the videos that play correctly have HDR10 metadata, included. So, obviously they will play correctly. The ones that don't play correctly, also don't have 'EL' in the HDR format.

[haasn]

@ValZapod ITU-R ICtCp != Dolby Vision IPT

[jeeb]

@ValZapod Dolby Vision Profile 5 is not standard ICtCp. Dolby made sure you cannot just implement a standard thing they published, and get their newest marketing buzzword implemented.

With regards to link, fun. Esp. the part about non-thread-safeness. Might be worth checking since it does contain references to reshaping.

Some research was done earlier into this stuff https://code.videolan.org/videolan/libplacebo/commit/f850fa2839f9b679092e721068a57b0404608bdc , which lead to semi-OK results, but also weird switches at certain frames. A la https://0x0.st/zIBI.png → https://0x0.st/zIBl.png
(this is based on Dolby's profile 5 test sample from developer.dolby.com/tools-media/sample-media/video-streams/dolby-vision-streams/)

[haasn]

Also note, we are missing three parts here: first, pivots and second level polinomials, second, NLQ (Nonlinear quantization) approximation, third, cross talk matrix! See

We're not missing the cross-talk matrix. We just bake it into the IPT coefficient matrix. (e.g. standard ITU-R ICtCp has a crosstalk of c=0.02 embedded in the standardized coefficients).

But yes, we're missing the polynomial reshaping bits. That's the stuff that there's no public documentation for, especially for the exact bitstream. (Though there is that one patent which provides an example bitstream that doesn't appear to match reality? Unless I'm mistaken)

If you can figure out how to get these coefficients into the decoding pipeline somehow (e.g. as side data) we can implement the polynomial bits with relative ease.

[igorzep]

@ValZapod Dolby Vision Profile 5 is not standard ICtCp.

Sure it is not. It was the basis for ICtCp. Dolby did good work at researching the color spaces but ICtCp adds the last bit - optimise the space so it better fits into the T/P range so all code points are fully utilised for BT2020 color space source.

Dolby made sure you cannot just implement a standard thing they published, and get their newest marketing buzzword implemented.

Looking at how specs are evolving it rather seems they simply released raw unfinished product, which was not acceptable to put into standards.

With regards to link, fun. Esp. the part about non-thread-safeness. Might be worth checking since it does contain references to reshaping.

Thanks :) The part about thread safety is probably outdated already and just left there undeleted from my first experiments on composing a DVp5 compatible file. :)

Some research was done earlier into this stuff https://code.videolan.org/videolan/libplacebo/commit/f850fa2839f9b679092e721068a57b0404608bdc , which lead to semi-OK results, but also weird switches at certain frames. A la https://0x0.st/zIBI.png → https://0x0.st/zIBl.png

I done mine back in 2018 and found pretty much information about the DVp5 internals (which on large part is identical to DVp7, which is pretty well documented in patents). The hardest part was the IPTPQc2 color space (which took a ~two months just to find out it's (semi-official) name so further docs could be found which specified the crosstalk and reshaping matrixes that were not embedded into the RGB to LMS matrix same way as is done with ICtCp).

I am doing DVp5 not for purpuses of decoding but for encoding (creating test patterns). But problems I face are exactly the same as the process is exact reverse. The progress so far... it is not ideal, I still miss a point on why encoded video look undersaturated on the TV. One guess about why is that the data is assumed to cover BT2020 color space and the TV is more like P3... And contrary to other DV formats DVp5 carries no data about mastering display color space so TV can't decide where to compress and where just trim. Still, the colours are right and I don't see any shifts (except of lowering saturation) on several ColorChecker patterns. So, in this sense my code is more correct then, so is worth checking and comparing approaches.

Dolby did good work

Please refrain from comments that could be interpreted as extremely offensive.