Native vs application parsing of in-band cues

[chrisn]

At TPAC 2019 we discussed UA vs application parsing of in-band cues.

To simplify cue handling for web applications, the UA should parse in-band cues and present them to the application as structured data. This means that implementations need to be able to parse a defined set of cue formats, which raises the following questions:

• Which cue formats should be standardised across implementations?
• How should new cue formats that are introduced in future be handled?
• How does a web application know whether a particular cue format will be handled by the UA, to decide whether to parse the cues itself?

Given that in-band cues may carry arbitrary application-specific data, and it's not reasonable to expect implementations to parse all possible cue formats, a mechanism for the web application to parse the cue message data will be needed. We would want this work to happen off the main thread, to prevent stalling the UI. Possible approaches:

• The web application parses media segments to extract in-band cues, prior to passing them to MSE appendBuffer(). This is what player libraries do today.
• The web application registers a cue parser script, to be called when the implementation encounters cues matching a given format or scheme identifier.