Threading

[sandersdan]

Chromium's WebCodecs implementation will offload decoding to a separate thread. We probably want to specify this behavior.

An alternative would be to specify a Worklet type for codec implementations, but this doesn't seem compatible with having a codec that uses many threads.

[padenot]

Both seem useful. Big apps ported from native or advanced users probably want a synchronous API and use their own threading synchronized via shared memory to keep caches hotter when doing something with the output afterwards. Other users probably want something a bit more ergonomic.

Parallelism inside Web Codec constructs will be some sort of fork-and-join, but that's useful for intra-frame parallelism, for encoding video.

[chrishtr]

I'm not sure it makes sense to expose the WebCodecs API to the main thread at all. Shouldn't it always be in a worklet or worker? Is there a compelling use case that requires the main thread?

[chcunningham]

The spec's processing model now makes the threading behavior explicit
https://wicg.github.io/web-codecs/#codec-processing-model

Control thread: where you make the encoder/decoder (be it in a window or worker)
Codec thread: where actual encoding/decoding is to occur. May actually be N threads, but may not be the control thread.

I'm not sure it makes sense to expose the WebCodecs API to the main thread at all. Shouldn't it always be in a worklet or worker? Is there a compelling use case that requires the main thread?

This question probably goes away with my comment above. Apps with lots of main thread activity and/or heavy UI (e.g. video conferencing) should still use WebCodecs in a worker to avoid contention with controlling the codec and painting their UI. But for other apps, Window is fine.

[aboba]

Some questions were raised about WebCodecs support for threading here.

[chcunningham]

Chromium's WebCodecs implementation will offload decoding to a separate thread. We probably want to specify this behavior.

This is now spelled out in the spec. https://w3c.github.io/webcodecs/#control-thread-and-codec-thread

Big apps ported from native or advanced users probably want a synchronous API...

We discussed this in the call w/ audio epxerts. It is true that many are coming from a synchronous API, but they concluded that this was not a requirement for them. We also discussed how our API's under the hood are often async and they stated that they did not desire for us to hide that from them at the JS layer. Please open a new issue for this if I've overlooked anything.

I'm not sure it makes sense to expose the WebCodecs API to the main thread at all.

This same question is under more active discussion in #211.