Question/Future request: ShakaPlayer audio and subtitle track detection (audioDescription, original, sdh)

[jakubjereczek]

Have you read the FAQ and checked for duplicate open issues?
I did.

Is your feature request related to a problem? Please describe.

At the beginning, I would like to point out that I am using ShakaPlayer in Chromecast CAF.
I would like to inquire about the technical capabilities of ShakaPlayer for detecting the following audio/subtitle tracks roles in a manifest.

• Audio description (audio description for the visually impaired),
• Audio Original Version, (source audio version)
• Subtitle Hearing impaired (sdh: subtitling for the deaf and hard of hearing)

Describe the solution you'd like

To extend the audio/subtitle ShakaPlayer API in order to obtain information about the above values.

I don't know what it looks like in DASH, but in HLS everything is available in the CHARACTERISTICS attribute in the manifest.
audio description: CHARACTERISTICS="public.accessibility.describes-video"
hearing impaired: CHARACTERISTICS="public.accessibility.transcribes-spoken-dialog,public.accessibility.describes-music-and-sound"
original language: CHARACTERISTICS="public.original-content"

Describe alternatives you've considered

No alternatives.

Additional context

In our application, we need to add a detailed description and specify the mentioned types above and I would like to know if it is technically possible using ShakaPlayer.

[jilles-sg]

It looks like the idea is that shaka.extern.Track contains the necessary information:

• "Audio description": roles or audioRoles (?) contains caption or commentary (?)
• Audio "Original Version": primary is true
• Subtitle "Hearing impaired": kind is caption (not subtitle)

[avelad]

CHARACTERISTICS are currently included within roles in HLS, is that not enough? (see https://github.com/shaka-project/shaka-player/blob/main/lib/hls/hls_parser.js#L2356 )

[jakubjereczek]

Probably it is, sorry if my question wasn't clear. I'm using CAF that have their unusual implementation of ShakaPlayer API. To sum up i'm asking whether shaka player technically detects the above "roles" (HLS was just an example).

In the company, we also use RxPlayer, which it provides the API for two of the three "roles" above.

• isAudioDescription (audio track has added commentaries for the visually impaired)
  type: Boolean|undefined
• isClosedCaption (if true it means that the text track has added hints for the hard of hearing).
  type: Boolean|undefined

[avelad]

I created #5819 for "Subtitle "Hearing impaired""

I have not been able to find the public.original-content in the HLS spec (https://developer.apple.com/streaming/HLS-draft-pantos.pdf) so we cannot accept this change. Regarding the change of "Audio description" I would like to know @theodab's opinion

[avelad]

@littlespex do you have any opinion about that?

[littlespex]

We use public.accessibility.describes-video to flag audio description tracks in HLS, to match DASH's <Accessibility schemeIdUri="urn:tva:metadata:cs:AudioPurposeCS:2007" value="1"/>.

[avelad]

@littlespex Thanks for your response, I will do the same in the HLS implementation