CEA 708 closed captions

[joeyparrish]

Write a plugin for CEA 708 closed captions.

[vincentma]

I have a curious question. Actually the CEA 608/708 captioning data lives inside the video data, in a specific way, frame NAL unit. So for a DRM protected stream, there's no way to see (or parse) the video data in application code.

So the ideal solution is to have the parser sitting inside the Chrome's video element decoding process, and populate the CC tracks.

Correct me if I am wrong.

[joeyparrish]

We will support unencrypted CEA 708 closed captions in their own stream, not muxed into the video stream. The video stream could still be encrypted.

[robwalch]

@joeyparrish any update on this? We're considering forking and providing a PR with this feature, but could use some pointers on implementing it as a plugin that has access to video data.

[TheModMaker]

We have not started work on this, but a PR would be appreciated. It would be best implemented as a plugin to TextEngine (see VttTextParser). The plugin will be given the media segment (as an ArrayBuffer) and it will produce an array of VTTCues.

[joeyparrish]

@robwalch What do you mean by "access to video data"?

[DanielBaulig]

@joeyparrish CEA-608/CEA-708 is usually embedded as SEI messages among the video samples. I assume that is what he means.

[joeyparrish]

In that case, you could create a network response filter that would have access to all segments before they are fed to MediaSourceEngine.

A text parser plugin would be needed to feed the cues to TextEngine.

But it between it may get a little ugly. Nothing in our architecture allows for text to be muxed with anything else.

In today's Shaka Player, our StreamingEngine needs to see that there is a text stream according to the manifest. Based on that, it will try to fetch text segments to feed to the text parser. So there's not a clear path at the moment.

[chrisfillmore]

@joeyparrish I have a question: is there a reason CEA-608 / CEA-708 support is tied in to HLS?

A coworker of mine who works on the video side was asking about Shaka support for these standards, and he didn't think it applied particularly to HLS (other than the fact that DASH specifies that text tracks should be included in the manifest). Can you clarify?

(We don't need this feature, I'm just asking out of curiousity. Thanks.)

[joeyparrish]

It's tied more to TS than to HLS. The HLS label is because TS-related functionality has been added almost exclusively for the sake of legacy HLS content libraries from before the addition of fMP4 to the HLS spec. Although TS is allowed in DASH, in practice, we haven't run into it in the wild.

[TheModMaker]

I don't think this is closed since we don't support CEA 708 on platforms with native TS support (e.g. Edge and Chromecast). At the very least we should add an option to force transmuxing so we can parse these captions. Alternatively we might be able to have mux.js just parse the captions and not actually transmux anything.

[joeyparrish]

Let's add an option, then, to force transmuxing for native TS platforms. It should default to false.

[aibolik]

Hi. Whom I can refer to get updates on the work progress of supporting CEA 608/708 captions for native TS support platforms, more precisely for Chromecast.

I see that Chromecast with CAF uses Shaka Player v 2.2.10, but the latests in the repository is 2.3 with some bug fixes and features. Does anyone know when new version will be added to CAF?

P.S.: don't you mind adding badge with the version? Thanks

[joeyparrish]

We are not in control of the CAF release schedule. I don't know when CAF will release with v2.3 or v2.4. This feature will first appear in v2.4 releases.

Can you clarify what you mean about a badge?

[theodab]

I have a CL in progress for that. It should be ready for review today.

[aibolik]

@joeyparrish I meant something like this, just a tiny thing.

[joeyparrish]

@aibolik, I still don't understand. Do you want a badge on the issue with the version in which it was fixed? Or do you want a badge on the main page with the current release?

[aibolik]

@joeyparrish exactly, badge with current release would be informative.

[joeyparrish]

You can always find the latest release at the top of this page: https://github.com/google/shaka-player/releases

[joeyparrish]

This feature will be out in v2.4 this week.