Deprecate VideoTrackReader

[tguilbert-google]

The MediaStreamTrack API for Insertable Streams of Media (a.k.a Breakout Box) provides equivalent functionalities to VideoTrackReader/VideoTrackWriter (+ audio equivalents). They both allow a web application to obtain VideoFrames from a MediaStreamTrack, or to create a MediaStreamTrack from a stream of VideoFrames.

Breakout Box was on track to ship eventually, regardless of WebCodecs status. We therefore looked into if it was worth keeping VideoTrackReader, considering that there would be two ways on the web platform of doing the same thing.

An initial investigation into whether or not there were advantages that VTR with no equivalents in MSP did not find anything. Setting a queue size was added to the MSP design as a result this investigation, to allows web applications to prioritize latency with small queues, or to prevent dropping frames with a larger queue.

VTR's advantage was the simplicity of its API: a callback based design that doesn't require users to know anything about the Streams API. However, we did not find that the code created by streams MSP looked particularly complicated (e.g. demo #1 or demo #2). Futhermore, the ability to transfer a stream endpoint (rather than posting each frame) makes for clean code in MSP's case (e.g demo #1 in workers).

Keeping VTR for the sole reason of offering a callback alternative to MSP is untenable. Additionally, MSP offers a distinct opportunity for platforms to implement under-the-hood optimizations, not available to VTR. Specifically, VTR must receive frames on the main thread and then post them to a worker; MSP can transfer a stream endpoint to a worker, and receive frames directly on the worker thread, thus helping main thread contention.

In short,

• VTR (and co.) doesn't offer any unique advantages over MSP (and co.), while MSP has unique better performance potential with workers
• MSP's use of streams is justified and shouldn't be much harder to use compare to VTR's callback
• Only having one way to do things on the web platform is healthier for the platform, and less maintenance overall

For these reasons, we are deprecate VideoTrackReader (and not developing VideoTrackWriter + audio equivalent), and instead helping out with the developement of MediaStreamTrackProcessor/MediaStreamTrackGenerator.

[tguilbert-google]

FYI, Chromium bug tracking the change: crbug.com/1157610