WebCodecs

[pthatcherg]

Hello TAG!

I'm requesting a TAG review of:

• Name: WebCodecs
• Specification URL: None yet (but there is early WebIDL)
• Explainer: https://github.com/WICG/web-codecs/blob/master/explainer.md
• GitHub issues: https://github.com/WICG/web-codecs/issues
• Tests: None yet
• Primary contacts: @steveanton, @padenot, @aboba, @pthatcherg

Further details:

• Relevant time constraints or deadlines: None
• I have read and filled out the Self-Review Questionnare on Security and Privacy. The assessment is here.
• I have reviewed the TAG's API Design Principles
• The group where the work on this specification is: @WICG
• Major unresolved issues are tagged as "hard" on the github issue tracker: https://github.com/WICG/web-codecs/issues?q=is%3Aopen+is%3Aissue+label%3Ahard

You should also know that...

• The review is early on in the design stage because we expect to learn a lot from early implementation feedback, so we'd like to get a rough design reviewed so that we can begin implementations and get that feedback to further refine the design.
• A portion of the explainer and WebIDL could be done a separate standard from WebCodecs, namely the conversions from WHATWG streams to MediaStreamTracks (called TrackWriters and TrackReaders at the moment). Many use cases require these, but use cases outside of WebCodecs could also find these useful. For simplicity, we are considering such work inside the scope of WebCodecs, although it could be pulled out later.

We'd prefer the TAG provide feedback as (please select one):

• open issues in our GitHub repo for each point of feedback

[dbaron]

I've read through the explainer (twice, I think) and I don't have anything to complain about so far.

[cynthia]

I've also looked at this and think it would be a useful addition to the platform. One question I do have is if the first deliverable could also support images - there is [1] HTMLImageElement.decode() but that really doesn't do exactly what one would expect. Could any of the editors comment on if this is possible?

CC: @steveanton, @padenot, @aboba, @pthatcherg

[1] https://developer.mozilla.org/en-US/docs/Web/API/HTMLImageElement/decode

[padenot]

I'm pretty sure this would be possible. More and more, image codecs use video codec technology/bitstream.

[plinss]

Given that these APIs offer more control than existing places where codecs are used, such as MediaRecorder, has any thought been given to exposing the existing codes via these APIs? For example, I can see starting a MediaRecorder, getting the video encoder and setting options (or swapping an encoder I construct), rather then extending the MediaRecorder APIs to have similar control surface...

[cynthia]

Raised w3c/webcodecs#50 to track the image use case.

[dbaron]

@cynthia and I are looking at this in a breakout at the TAG's virtual face-to-face.

We (the TAG) have looked at this a few times, and I think we're generally pretty happy with it. The design looks appealingly simple -- which is either great (if it can stay simple in the end), or it's a sign that a bunch of things were missed in the design. We're hoping it's the former.

Peter's comment gets at one of the tricky issues with adding new fundamental APIs that other things are built on, when those other things already exist. When that happens, there's generally a process of making the existing features, like MediaRecorder, work with the new fundamental API. That doesn't need to happen all at once, but it does often require somebody making sure that progress happens (e.g. by filing appropriate issues on places that will need to be changed, so we don't lose track) so that the desired end state can be reached. We hope that happens here once the foundation is ready.

So I think we're going to close this issue. Thanks requesting our feedback. Let us know if there's anything else you'd like us to look at later on in the process.