Caching issue

[Antonito]

Your environment.

• Version: a43306d
• Browser: Not relevant
• Other Information: https://github.com/Antonito/gfile

What did you do?

I'm using pions/webrtc datachannels in a project to do direct file sharing.
The file transfert is stable, but the speed drops heavily over time.
I'm using a datachannel initialized like this:

ordered := true
maxPacketLifeTime := uint16(10000)
dataChannel, err := s.peerConnection.CreateDataChannel("data", &webrtc.DataChannelInit{
	Ordered:           &ordered,
	MaxPacketLifeTime: &maxPacketLifeTime,
})

What did you expect?

Transmission speed shouldn't decrease that much over time.

What happened?

The file transfer is stable, but the speed decreases over time, making it almost unusuable for long file transferts.
After cpu/mem profiling the issue, I'm pretty confident the issue is that abandoned packets are never removed from the orderedPackets (payloadQueue, payload_queue.go). My program spends ~80% of its time in the Association.sendPayloadData method (association.go), and more specifically ~60% of its time sorting the orderedPackets array.

I'm not sure if this behavior is due to a bug or due to a RFC spec not built-in yet.

Here is the profiling datas of a 50MB file transfert:
cpu-profiling.pdf
mem-profiling.pdf

[enobufs]

Thanks @Antonito - it was nice chatting with you earlier.

Since current SCTP does not have congestion control, it tries to send all the pending data pushed by the application. As network bandwidth is finite, I can image lots of packet loss would be happening if you push large data too fast (probably, the CPU is doing large number of retransmissions also).

Even with proper congestion control is implemented, it is still the application's responsibility to control how fast data has to be pushed to the data channel, which is typically done by monitoring a parameter called "bufferAmount" which is, again, not available yet.

https://developer.mozilla.org/en-US/docs/Web/API/RTCDataChannel/bufferedAmount

If you push large data at once, SCTP layer would be busy working on appending and sorting the send buffer (payloadQueue), so I can imagine much CPU time is spent for sorting the data a lot.

Once the congestion control is implemented, we should be able to enable bufferedAmount (and other related threshold) parameter, with which the application can control the amount of data to be pushed to the channel.

I will also looking into more efficient way to manage buffered chunks. (append / sorting arrays may not be super efficient).

Relates to #11