Access to raw metadata

[youennf]

I can see use cases where it would be useful to have metadata attached to raw frames be exposed to JavaScript.
This could for instance be head detection metadata that some cameras already generate.
Or it could be a track processor that adds such metadata in addition to (or in lieu of) modifying the pixels.

There may be also use cases where this metadata should be preserved by the encoder (or decoder) so that the JS raw frame metadata would be kept attached to the JS encoded frame (or vice versa).

[alvestrand]

Absolutely agree. But given that we are adopting the frames from WebCodecs, can you raise an issue on this in the WebCodecs repo and link it here?

[alvestrand]

@youennf ?

[youennf]

Thanks for the ping, will do it shortly.

[alvestrand]

Great, closing in this repo.

[youennf]

I think there are WebRTC bits we need to discuss outside of WebCodecs.
In particular, we want that metadata added by media capture raw goes up to WebRTC encoded transform, which is not WebCodec.

Reopening this issue for now. Though maybe we could move it to webrtc encoded transform.

[guidou]

I think we have been discussing this in the context of encoded transform, so closing this.