Add transfer-functions and hdr-metadata to video-capabilities.

[mfoltzgoogle]

Addresses Issue #194: [Remote Playback] Capabilities for HDR rendering and display

This adds two fields to the receive-video-capability message that allows the receiver to list the HDR metadata types and transfer functions that the media receiver can accept.

The values are used by reference from the corresponding enums in the Media Capabilities API. This was discussed, briefly, in the minutes of the joint SSWG/Media WG meeting:

https://www.w3.org/2022/01/11-mediawg-minutes.html#t01

---

Preview | Diff

[mfoltzgoogle]

One question that could use further input is whether the two new values are independent, i.e. if you have:

transfer-functions: [  "p3", "rec2020"  ]
hdr-metadata: [ "smpteSt2086", "smpteSt2094-10" ]

Is it always true that all four combinations are possible? If not, then we might have to adjust the data model to be a list of transfer-function and hdr-metadata pairs.

Probably requires looking at some HDR displays to see what the constraints are in the real world.

[mfoltzgoogle]

After reviewing some of the relevant documentation around the three linked fields color-gamuts, transfer-functions and hdr-metadata, here is what I think makes sense for the protocol.

• The color gamut values are inclusive by definition: rec2020 includes p3, which includes srgb. We could change color-gamuts from a list to a single field, as long as we made it clear that the agent receiving the color-gamut applied the inclusion logic. I will file an issue for this, but I don't feel strongly that this requires a change to the protocol.

• Some hdr-metadata and transfer-functions values don't make sense. For example HLG does not require a separate metadata format, so a media receiver that understood both e.g. HDR10+ and HLG via one codec would have to report them as two separate encoding types with all other fields duplicated:

Encoding 1:

<repeat everything else>
hdr-metadata: []
transfer-functions: ['hlg']
<repeat everything else>

Encoding 2:

<repeat everything else>
hdr-metadata: ['smpteSt2094-40']
transfer-functions: ['pq']
<repeat everything else>

For this reason I think we should report the HDR capabilities as a list of pairs, so this can be conveyed in one encoding:

<snip>
hdr-formats: [
  { transfer-function: 'hlg' },
  { transfer-function: 'pq', hdr-metadata: 'smpteSt2094-40' }
]
<snip>

meaning that encoding could convey either HLG or HDR10+. For completeness, here is what it would look like with smpteSt2094-10 included, which can support either HLG or PQ [1].

<snip>
hdr-formats: [
  { transfer-function: 'hlg' },  // HLG with no metadata
  { transfer-function: 'hlg', hdr-metadata: 'smpteSt2094-10' }  // 2094-10 with HLG. [2]
  { transfer-function: 'pq', hdr-metadata: 'smpteSt2094-10' }   // 2094-10 with PQ
]
<snip>

Note, per the issue linked below, Dolby has registered a number of MIME types to represent their proprietary HDR metadata encodings, so remote playback capability detection for these formats would be handled by the remote-playback-source message, which accepts full MIME types.

[1] w3c/media-capabilities#136
[2] Unclear if if this happens in practice as HLG support may be a proprietary profile of Dolby Vision and not part of smpteSt2094-10.

[mfoltzgoogle]

@chrisn, we have resolved the issues preventing PRs from landing on the OSP spec. Feel free to take a look.

[chrisn]

Thank you, somehow I missed your previous comment here, I'll take a look.

[chrisn]

We could change color-gamuts from a list to a single field, as long as we made it clear that the agent receiving the color-gamut applied the inclusion logic. I will file an issue for this, but I don't feel strongly that this requires a change to the protocol.

I agree.

For this reason I think we should report the HDR capabilities as a list of pairs

This sounds good.

Dolby has registered a number of MIME types to represent their proprietary HDR metadata encodings, so remote playback capability detection for these formats would be handled by the remote-playback-source message, which accepts full MIME types.

I see. If we need to we can follow this up with them via the new A/V Media Formats CG, which started at TPAC 2022.

[mfoltzgoogle]

We could change color-gamuts from a list to a single field, as long as we made it clear that the agent receiving the color-gamut applied the inclusion logic. I will file an issue for this, but I don't feel strongly that this requires a change to the protocol.

I agree.

Filed #305

[mfoltzgoogle]

For this reason I think we should report the HDR capabilities as a list of pairs

This sounds good.

Ok, updated PR to reflect this change.

[mfoltzgoogle]

Feel free to review the current PR @chrisn with the changes I proposed, however, the spec generation is currently broken again so you won't be able to see an HTML preview yet. ☹️

[chrisn]

Thanks. I've asked a colleague for advice on this part:

[2] Unclear if this happens in practice as HLG support may be a proprietary profile of Dolby Vision and not part of smpteSt2094-10.

[mfoltzgoogle]

Hi @chrisn, did you have any additional feedback on the PR?

Regarding:

[2] Unclear if this happens in practice as HLG support may be a proprietary profile of Dolby Vision and not part of smpteSt2094-10.

Would definitely appreciate any feedback on this. However, the protocol proposed in the PR should support this use case if it is necessary.

[chrisn]

This is good, thanks!

[mfoltzgoogle]

Thanks!