TextTrackCue enhancements for programmatic subtitle and caption presentation

[marcoscaceres]

Request for Mozilla Position on an Emerging Web Specification

• Specification title: TextTrackCue enhancements for programmatic subtitle and caption presentation
• Specification or proposal URL (if available): http://github.com/whatwg/html/pull/9771
• Explainer URL (if available): https://github.com/WebKit/explainers/tree/main/texttracks
• Proposal author(s) (@-mention GitHub accounts): @marcoscaceres @hober @eric-carlson
• Caniuse.com URL (optional):
• Bugzilla URL (optional): https://bugzilla.mozilla.org/show_bug.cgi?id=1855236
• Mozillians who can provide input (optional): @zcorpan @emilio (was unsure who to ping here... maybe one of you can ping the right folks? 🙏)

Other information

Native UA rendering of out-of-band subtitles and captions is currently only possible with WebVTT. If a site can't or won't make its subtitles available in-band, and can't or won't publish its subtitles and captions in WebVTT, the site must render its subtitles and captions itself.

This proposal aims to:

• Allow video publishers to continue storing captions in legacy or custom formats.
• Make sure users can control the presence and style of captions with system accessibility settings.
• Allow captions to be served in custom formats, but still be rendered by the browser engine.

We (WebKit) would like to make the following changes to HTML:

• Exposes a constructor for TextTrackCue which takes startTime, endTime, and cueNode arguments.
• Defines authoring and processing requirements for cue fragments
• Adds cue and cuebackground global attributes.
• Add infrastructure for selecting ::cue and ::cue-background pseudo elements, which can then be styles by the CSS or the system.

[zcorpan]

@padenot

[zcorpan]

@alastor0325

[alastor0325]

@marcoscaceres Hello Marcos, I've checked the explainer and most of them look good to us as well. The only thing I am still curious is that why we need to extend the constructor of TextTrackCue, instead of keeping using (or modify) the constructor of VTTCue that already accepts start/endtime?

If I understand that correctly, the change is intended to ask video publishers to use TextTrackCue to add their captions, which are converted from their legacy or custom formats, but they can already do that via adding VTTCue already. Thanks!

[marcoscaceres]

Thanks @alastor0325 for the feedback.

If I understand that correctly, the change is intended to ask video publishers to use TextTrackCue to add their captions, which are converted from their legacy or custom formats, but they can already do that via adding VTTCue already. Thanks!

That is correct. The difference with the new proposal is that it allows for a slightly expanded set of tags to be used (e.g., <p> and <div>). WebVTT currently won't support that, unless we also expand the WebVTT format and parser, which is probably not something we want to do.

So the idea is to be slightly less restrictive than WebVTT.

[marcoscaceres]

So, we thought about the some more and concluded that we should just fix WebVTT. It would be great if we collectively get this into Iterop 2025, as WebVTT has a lot of interop issues... once we get those fixed (or to a better state), it might be worth returning to this.