Add support for compositing a TTML2 document onto HDR video

[palemieux]

Colors in TTML2 documents are specified using sRGB. This yields ambiguities when compositing such documents onto HDR video, whose allows peak luminance much higher than that associated with sRGB, e.g. ITU Rec 2100 allows luminance values up to 10000 cd/m^2 while sRGB specifies a reference white of 80 cd/m^2.

As a practical example, white sRGB caption text should rarely, if ever, be composited onto HDR video assuming that sRGB (255,255,255) maps to peak HDR white: the resulting text would be unacceptably brighter than the rest of scene, whose average luminance is typically far below peak HDR white. At what luminance level should sRGB captions be composited onto HDR images -- without having to define new colorspaces and encodings for TTML2?

To allow authorial control of the compositing of TTML2 documents onto HDR images, the attached proposes an tts:hdrAbsoluteLuminanceGain attribute, which allows the normalized rgb components of each pixel of a region to be mapped to absolute optical output components (in cd/m^2)

(R, G, B)= 80 cd/m^2 ∙ hdrAbsoluteLuminanceGain ∙ (r, g, b)

It would also be useful to note that, in the case of image-based subtitles, tts:tts:hdrAbsoluteLuminanceGain is not necessary if the image is an HDR image.

[skynavga]

On Wed, Jan 11, 2017 at 4:55 PM, Pierre-Anthony Lemieux < ***@***.***> wrote: Colors in TTML2 documents are specified using sRGB. This yields ambiguities when compositing such documents onto HDR video, whose allows peak luminance much higher than that associated with sRGB, e.g. ITU Rec 2100 allows luminance values up to 10000 cd/m^2 while sRGB specifies a reference white of 80 cd/m^2. As a practical example, white sRGB caption text should rarely, if ever, be composited onto HDR video assuming that sRGB (255,255,255) maps to peak HDR white: the resulting text would be unacceptably brighter than the rest of scene, whose average luminance is typically far below peak HDR white. At what luminance level should sRGB captions be composited onto HDR images -- without having to define new colorspaces and encodings for TTML2? The following proposes to add a tts:hdrAbsoluteLuminance attribute, which allows the author to specify the peak white luminance, i.e. the absolute luminance of the sRGB triplet (255, 255,255), when compositing a region on an HDR image: - tts:hdrAbsoluteLuminance applies to only - the value of tts:hdrAbsoluteLuminance is a decimal number between 0 and 100 - the initial value of tts:hdrAbsoluteLuminance is 1 - the peak white luminance is equal to 100 * tts:hdrAbsoluteLuminance cd/m^2 It would also be useful to note that, in the case of image-based subtitles, tts:hdrAbsoluteLuminance is not necessary if the image is natively stored as an HDR image.

just a few comments at this time: - should be defined in ttp rather than tts namespace, since it is effectively a processing parameter that applies to entire document, i.e., would/should not vary within document; - specification text would have to weigh in on handling of HDR images, i.e., can't just say nothing —

…

You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#233>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAXCb3YmopXZZ9RVTZ71v7aPJ3UjxJQqks5rRWvzgaJpZM4LhNxV> .

[palemieux]

• should be defined in ttp rather than tts namespace, since it is
  effectively a processing parameter that applies to entire document, i.e.,
  would/should not vary within document;

The parameter would apply to individual regions and would change across the timeline of the document, to match the underlying HDR video.

specification text would have to weigh in on handling of HDR images, i.e., can't just say nothing

Ok. Are you thinking of listing, as examples, the concrete transformations involved in compositing sRGB + tts:hdrAbsoluteLuminance onto an HDR image?

[simontWork]

It should be noted that ITU-R BT.2100 contains 2 signal specifications for HDR video, one of which is based on absolute light (PQ) and one of which is based on relative light (HLG). Any standard for the display of subtitles should contain methods for both systems.

For the relative system, a luminance mapping in either the linear scene light or linear display light domain, and a colour gamut conversion is required. One method of doing this is as follows:

• Convert sRGB to CIE Yu'v'
• Scale Y (and adjust gamma for critical, non-graphics applications)
• Convert CIE Yu'v' to RGB2020_NCL
• Apply HLG opto-electronic transfer function

[palemieux]

@simontWork This proposal is intended to address compositing onto absolute luminance images. To address compositing on HLG images, either (i) a separate ticket should be filed or (ii) concrete tweaks to this proposal suggested (assuming a single parameter can handle both).

[nigelmegitt]

@palemieux Given that this issue is non-specific about the HDR system, I think @simontWork 's comments are entirely relevant here, and actually rather useful since they suggest that no further parameters or style attributes are required to support relative system HDR.

[palemieux]

@nigelmegitt I am not sure what @simontWork is proposing then. Perhaps you can shed some light?

[simontWork]

@palemieux For each colour used for subtitle text in the SDR domain, a simple mapping can be applied to provide RGB or YCbCr co-ordinates in the relative HDR domain, so there is no need for metadata (apart from signalling of the subtitle colour format if more formats than sRGB format subtitles are available).

[palemieux]

For each colour used for subtitle text in the SDR domain, a simple mapping can be applied to provide RGB or YCbCr co-ordinates in the relative HDR domain,
so there is no need for metadata (apart from signalling of the subtitle colour format if more formats than sRGB format subtitles are available).

@simontWork Ok. Great. Are you suggesting that the One method of doing this is as follows be included in the specification? If so, can you provide a detailed mathematical walkthrough.

[simontWork]

@palemieux @nigelmegitt If we could have a placeholder in the document, we will draft the reply ASAP

[palemieux]

Issues description updated with additional details.

[palemieux]

@simontWork Any update on a proposal?

[nigelmegitt]

@palemieux BBC is working on a simplified proposal.

[palemieux]

What are the differences, if any, between the proposal and the edited version?

[skynavga]

The only non-editorial difference is that the name of the attribute was shortened to tts:luminance.

…

On Thu, Mar 30, 2017 at 1:22 AM, Pierre-Anthony Lemieux < ***@***.***> wrote: What are the differences, if any, between the proposal and the edited version? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#233 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAXCb3jdY8MTE0R3NsbcGtlSrSnQfTt1ks5rq1gmgaJpZM4LhNxV> .

[palemieux]

in tts:hdrAbsoluteLuminanceGain,

• absolute is intended to differentiate from HDR schemes that do not use absolute luminance values
• gain is important since the value is not luminance, i.e. does not have units of luminance, but instead a uniteless gain applied to luminance (current industry practice)
• hdr is important since the gain should not be applied unless compositing to HDR images

So the name should remain.

[nigelmegitt]

In #233 (comment) @palemieux articulated very well exactly the position that the BBC shares on this. The attribute needs to be renamed back to tts:hdrAbsoluteLuminanceGain or something else that carries a similar set of connotations, and the specification for it also needs to reflect that it should not be applied in all situations.

As per previous comments, we are still working towards providing an alternative algorithm that is usable in other HDR scenarios. This is currently going through our internal processes for reasons that will become clearer later.

[skynavga]

irrelevant; we don't and will not have any other style properties related to luminance, so there is absolutely no need for the longer name; it is just a name, and the editor claims editorial privilege on naming, something I have done in prior versions of TTML on a number of occasions;

…

On Thu, Mar 30, 2017 at 5:07 AM, Pierre-Anthony Lemieux < ***@***.***> wrote: in tts:hdrAbsoluteLuminanceGain, - absolute is intended to differentiate from HDR schemes that do not use absolute luminance values - gain is important since the value is not luminance, i.e. does not have units of luminance, but instead a uniteless gain applied to luminance (current industry practice) - hdr is important since the gain should not be applied unless compositing to HDR images So the name should remain. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#233 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAXCbzEgrPci9RX7nD95tGCKGrR19fb4ks5rq40OgaJpZM4LhNxV> .

[palemieux]

See TTWG minutes at https://www.w3.org/2017/03/30-tt-minutes.html

[palemieux]

See #283 (comment)

[nigelmegitt]

Meeting 2017-04-20: Agreed to a working assumption of the name luminanceGain to be adopted via a PR to be raised by @palemieux . Also @nigelmegitt will propose an additional PR to add HLG support.

[nigelmegitt]

BBC (thanks especially to @simontWork but also others who know who they are) submits the following document containing a proposal for an addition to Appendix P illustrating a simple method for converting sRGB to HLG for HDR:

SimpleConversion_sRGBtoHLG[9].pdf

This algorithm does not make use of the tts:luminanceGain style attribute.

I will generate a pull request based on this proposal.

Update This submission is updated at #397 (comment) to fix a bug found within it.

[nigelmegitt]

I split out the HLG work into #298 to make it easier to track the editing work for that separately.

[skynavga]

Merged #294 and closing.