add mimeType to metadata

[fippo]

since they mapping from payload type to the codec mime type
is nontrivial in environments like workers.

Defined similar to webrtc-stats:
https://w3c.github.io/webrtc-stats/#dom-rtccodecstats-mimetype

Fixes #158

---

Preview | Diff

[youennf]

since they mapping from payload type to the codec mime type
is nontrivial in environments like workers.

The mapping should be quite stable so it is reasonably straightforward for web applications to postMessage SDP to script transforms.
Which cases are you thinking of for this API? Or is it just to avoid sending the SDP?

[fippo]

The use-case is codec-adaptive encryption schemes that can differentiate between VP8, VP9 and H264 for example, you will remember this long-standing issue.

Doing a mapping in JS requires parsing SDP and keeping a copy of the SDP around and synchronized in the worker at all times. And would be nontrivial for unbundled cases where you can have more than one codec associated with a payload type (even though I bet this case isn't well defined in stats mimeType).

At the sender side the information what encoder was used is readily available since the frame comes from there. At the receiver side one would assume the same but it turned out to be a bit more complicated to implement.

[alvestrand]

Needs a bug and resolution in the WG.

[fippo]

added a slide to the deck

[alvestrand]

Filed #158 in order to have a corresponding issue.

[fippo]

definition added. Now the definition is rather vague but we can improve that both here and in -stats later.

[aboba]

Resolution from 10-18-2022 WG meeting: "Add mimeType to Metadata".

[solmaks]

@youennf , @alvestrand , seems that WG agreed to add mimeType to metadata. Anything preventing from merging this PR?

[alvestrand]

biggest hole in spec is whether this is audio/opus (naked mime type) or audio/opus;channels=2;rate=16000;stereo=1 (full mime type, including the info from a=fmtp).

Since I've been in the camp of "naked mime type is incomplete" since forever, I'm of course in the camp of the second option.

[fippo]

At least for use-cases like end-to-end encryption the "raw" mimeType is sufficient, you want to know whether to expect VP8 or H264 (not that we have any specification of the actual behavior of current implementations)

I do agree that having the fmtp would be good in addition.

[alvestrand]

Discussion at December interim was inconclusive on whether we want alignment with WebCodecs (string with parameters), with PeerConnection (base type + fmtp line), or something else.

[alvestrand]

Suggestion: Use the same format as RTCRtpCodecCapability (consistency with webrtc-pc).

[fippo]

In that case I suggest not merging this until there is an implementation.

While I understand the desire for consistency I do not see how to satisfy this additional requirement nor do I know what additional problem this solves.

[youennf]

The suggestion is to stick with a raw mimeType for now, which is what this PR is proposing.
If we later want to expose the fmtp information, we would add a separate sdpFmtpLine member.
How about proposing this at next interim to see whether that reaches consensus?

[fippo]

Updated.

[alvestrand]

@fippo can you rebase again?

[fippo]

see #204 for validation failures which seem unrelated.

[dontcallmedom]

#208 would fix the validation error reported by the CI on this PR

[alvestrand]

Somehow this PR now touches "timestamp" in a way that causes the builder to fail.

[dontcallmedom]

@alvestrand you would have to rebase to get the CI error to disappear

[fippo]

squashed and rebased