Backpressure/buffering for hardware decoders?

[mfoltzgoogle]

The Streams spec assumes you can get some backpressure signal from the implementation of a TransformStream.

But a hardware codec may not give you that much control over what happens after frame data is passed into the codec API. It will typically decode the frame data immediately and render it into a GPU frame buffer.

So the high level feedback is: the spec (at a future level of maturity) should map the behavior of an abstract codec onto the behavior of the TransformStream, and allow codec implementations that run in immediate mode (without buffering). (Or require implementations to do this buffering internally.)

[steveanton]

I think we'd require the implementation to do the buffering internally. I suspect browsers have general frameworks for implementing Streams that can do this (the Streams spec describes it).

@pthatcherg could we get a "spec language" label?

[sandersdan]

My current implementation estimates the number of frames buffered internally by the codec (assuming one input chunk = one output chunk), and tries to keep internal buffered frames + output buffered frames below a constant.

It'll need more tuning to be production ready but it seems workable.

[padenot]

I believe this is handled via (e.g.) the decodeQueueSize attribute (and its encode counterpart of course). @sandersdan, do you feel this addresses the concerned expressed here?

[sandersdan]

Yes, decodeQueueSize is adequate, and was designed to fit will well with the streams backpressure model.

There is more we could do (a few different statistics, an event when space becomes available), but decodeQueueSize seems fine for V1.

[chcunningham]

For new queuing stats/events, I vote to track separately should requests for those arise.

Otherwise seems like discussion here is concluded. Please re-open if I've overlooked anything.