Refactor data consumer SCTP stream parameter handling

[threema-lenny]

The API of DataConsumer and DataProducer have been unified to use the same kind of parameters object when providing SCTP stream parameters and those parameters are always validated.

Previously, creating a DataConsumer from a DataProducer that has been created via DirectTransport was brittle since SCTP stream parameters cannot be inferred from the DirectTransport's DataProducer.

It is now possible to provide the SCTP stream id for the consumer, too.

---

My primary motivation was to be able to set the SCTP stream id for the consumer and then I noticed that the current API is a bit brittle when used in combination with a DirectTransport, leaving me with an SCTP DataConsumer whose parameters were stuck at default and didn't match the original DataProducer.

I'm not sure how controversial this API change is but I'm certain you will tell me. 🙂

(This builds on #1110, so I'm marking it as a draft for now.)

[ibc]

There are many concerns in this PR:

1. As told in Replace clone in utils with structuredClone everywhere #1110 (comment) we cannot require Node > 16 yet so we cannot rely on structuredClone().
2. This PR is literally changing the public API of the transport.consumeData() public method, so it's breaking the API. We cannot break the v3 API.
3. This PR doesn't address the corresponding changes in the Rust layer.

Regarding this:

Previously, creating a DataConsumer from a DataProducer that has been created via DirectTransport was brittle since SCTP stream parameters cannot be inferred from the DirectTransport's DataProducer.

If you create a DataProducer in a DirectTransport then there is litter you can configure. By definition the communication through a DirectTransport is ordered and reliable so no message can be lost and messages are guaranteed to be delivered in order. And hence, if you consume such a DataProducer, the settings of the DataConsumer will be fixed (reliable and ordered DataConsumer).

BTW the reliability of DataChannel in mediasoup is hop by hop and not end to end. If you create a reliable DataProducer on a WebRtcTransport and then consume it from another WebRtcTransport, both datachannels (uplink and downlink) will be reliable and ordered. However messages can be lost in the consuming side in case the consumer device cannot handle the traffic. This is a known limitation and something we'll reconsider in mediasoup v4.

[threema-lenny]

No worries, I totally get it if a major API change is not acceptable for you at the current point in time. My primary concern was the ability to add the SCTP stream id. Aligning the options objects with the one for the producer was secondary.

So, would you accept another PR instead that leaves the the current API as-is but adds the SCTP stream id as an optional parameter to the consumeData method?

If you create a DataProducer in a DirectTransport then there is litter you can configure. By definition the communication through a DirectTransport is ordered and reliable so no message can be lost and messages are guaranteed to be delivered in order. And hence, if you consume such a DataProducer, the settings of the DataConsumer will be fixed (reliable and ordered DataConsumer).

Thanks for the clarification! Then I'm assuming I misinterpreted that a consumer can consume unreliably from a reliable producer (because technically that should be possible - not that I would have a use case for this).

BTW the reliability of DataChannel in mediasoup is hop by hop and not end to end. If you create a reliable DataProducer on a WebRtcTransport and then consume it from another WebRtcTransport, both datachannels (uplink and downlink) will be reliable and ordered. However messages can be lost in the consuming side in case the consumer device cannot handle the traffic. This is a known limitation and something we'll reconsider in mediasoup v4.

Thanks for that insight. This made me curious. Can you elaborate where the "cannot handle" part could happen in the mediasoup code?

[ibc]

So, would you accept another PR instead that leaves the the current API as-is but adds the SCTP stream id as an optional parameter to the consumeData method?

Yes. But please don't do it in v3 branch but in flatbuffers branch to avoid conflicts when we merge its corresponding PR #1064 into v3 branch. Also, changes must also be done in Rust, and tests must be added in both.

Can you elaborate where the "cannot handle" part could happen in the mediasoup code?

If the network of the consuming device is poor and usrsctp lib cannot deliver all messages to it, then it will discard some of them and mediasoup will report the corresponding warning/error notification to the app.

[threema-lenny]

Yes. But please don't do it in v3 branch but in flatbuffers branch to avoid conflicts when we merge its corresponding PR #1064 into v3 branch. Also, changes must also be done in Rust, and tests must be added in both.

Sure, in that case I think I might wait until you have merged #1064.

If the network of the consuming device is poor and usrsctp lib cannot deliver all messages to it, then it will discard some of them and mediasoup will report the corresponding warning/error notification to the app.

Ah, is that the case where Router::OnTransportDataProducerMessageReceived does not set the onQueuedCallback and therefore never knows when the message was discarded down in SctpAssociation::SendSctpMessage? If so, I understand the design choice dilemma. Personally, we worked around it by shoving the data into the SCTP DataConsumers directly and simply backlogging the data in Node space in case of an error.

[ibc]

Ah, is that the case where Router::OnTransportDataProducerMessageReceived does not set the onQueuedCallback and therefore never knows when the message was discarded down in SctpAssociation::SendSctpMessage? If so, I understand the design choice dilemma. Personally, we worked around it by shoving the data into the SCTP DataConsumers directly and simply backlogging the data in Node space in case of an error.

usrsctp lib decides whether it can send the message or not, number of re-attempts, etc based on the quality of the current transmission. We, in mediasoup code, have no control about that. Yes, doing data buffering in Node layer is an approach. We use it as well.