Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement more RTCP stuff #29

Closed
ibc opened this issue Jun 20, 2016 · 21 comments
Closed

Implement more RTCP stuff #29

ibc opened this issue Jun 20, 2016 · 21 comments
Assignees
@ibc @jmillan
Milestone
v1

Comments

@ibc
Copy link
Member

ibc commented Jun 20, 2016 •
edited
Loading

mediasoup will not relay RTCP packets but consume and generate them. This is, mediasoup will act as a media endpoint by negotiating and managing RTCP with each client.
@ibc ibc added this to the 2.0.0 milestone Jun 20, 2016
@ibc ibc assigned ibc and unassigned ibc Jun 20, 2016
@ibc
Copy link
Member Author

ibc commented Nov 26, 2016

Lof of RTCP stuff done by @jmillan in #41

@ibc
Copy link
Member Author

ibc commented Dec 30, 2016

NACK processed! Habemus RTP retransmission.

@ibc ibc removed this from the 2.0.0 milestone Jan 2, 2017
@ibc
Copy link
Member Author

ibc commented Jan 10, 2017 •
edited
Loading

Among with other tasks, we must also define which exact RTCP feedback parameters mediasoup supports. For example, current browsers signal this:

  • Chrome:
m=audio 51753 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 126
a=mid:audio
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1
a=rtpmap:103 ISAC/16000
a=rtpmap:104 ISAC/32000
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:106 CN/32000
a=rtpmap:105 CN/16000
a=rtpmap:13 CN/8000
a=rtpmap:126 telephone-event/8000

m=video 64460 UDP/TLS/RTP/SAVPF 100 101 107 116 117 96 97 99 98
a=mid:video
a=rtcp-rsize
a=rtpmap:100 VP8/90000
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=rtcp-fb:100 goog-remb
a=rtcp-fb:100 transport-cc
a=rtpmap:101 VP9/90000
a=rtcp-fb:101 ccm fir
a=rtcp-fb:101 nack
a=rtcp-fb:101 nack pli
a=rtcp-fb:101 goog-remb
a=rtcp-fb:101 transport-cc
a=rtpmap:107 H264/90000
a=rtcp-fb:107 ccm fir
a=rtcp-fb:107 nack
a=rtcp-fb:107 nack pli
a=rtcp-fb:107 goog-remb
a=rtcp-fb:107 transport-cc
a=fmtp:107 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
  • Firefox:
m=audio 52876 UDP/TLS/RTP/SAVPF 109 9 0 8
a=fmtp:109 maxplaybackrate=48000;stereo=1;useinbandfec=1
a=mid:sdparta_0
a=rtpmap:109 opus/48000/2
a=rtpmap:9 G722/8000/1
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000

m=video 53804 UDP/TLS/RTP/SAVPF 120 121 126 97
a=fmtp:126 profile-level-id=42e01f;level-asymmetry-allowed=1;packetization-mode=1
a=fmtp:97 profile-level-id=42e01f;level-asymmetry-allowed=1
a=fmtp:120 max-fs=12288;max-fr=60
a=fmtp:121 max-fs=12288;max-fr=60
a=mid:sdparta_1
a=rtcp-fb:120 nack
a=rtcp-fb:120 nack pli
a=rtcp-fb:120 ccm fir
a=rtcp-fb:120 goog-remb
a=rtcp-fb:121 nack
a=rtcp-fb:121 nack pli
a=rtcp-fb:121 ccm fir
a=rtcp-fb:121 goog-remb
a=rtcp-fb:126 nack
a=rtcp-fb:126 nack pli
a=rtcp-fb:126 ccm fir
a=rtcp-fb:126 goog-remb
a=rtcp-fb:97 nack
a=rtcp-fb:97 nack pli
a=rtcp-fb:97 ccm fir
a=rtcp-fb:97 goog-remb
a=rtpmap:120 VP8/90000
a=rtpmap:121 VP9/90000
a=rtpmap:126 H264/90000
a=rtpmap:97 H264/90000

@jmillan
Copy link
Member

jmillan commented Jan 22, 2017 •
edited by ibc
Loading

Below are the RTCP types we are supporting.

For Feedback type "ack" or possitive acknowledgment:

  • ack rpsi (bypassed)
  • ack app (bypassed)

For Feedback type "nack" or negative acknowledgment:

  • nack (locally consumed)
  • nack pli (bypasssed)
  • nack sli (bypassed)
  • nack rpsi (bypassed)
  • nack app (bypassed)

For Feedback type "ccm" codec control:

  • ccm fir (bypassed)

The following, which are shown in the previous SDPs, are not supported:

  • goog-remb
  • transport-cc (this one looks promising BTW, but until we generate it we better remove the support from the SDP to avoid a ton of Feedback coming from Chrome.

@ibc
Copy link
Member Author

ibc commented Jan 22, 2017

Thanks! I will work on properly setting the supported values.

@ibc
Copy link
Member Author

ibc commented Jan 25, 2017 •
edited
Loading

@jmillan I'm pretty sure you don't need it. However FYI "JSEP" draft also defines RTCP routing in Appendix B.

@ibc
Copy link
Member Author

ibc commented Feb 6, 2017 •
edited
Loading

Next actions:

  • Improve/implement proper generation timing for Receiver Reports.
  • Locally generate Sender Reports.
  • Generate NACK for senders.
  • Generate PLI if we detect uplink heavy packet loss.
  • Improve NACK generation algorithm: Improve NACK generation algorithm #72
  • Move the RTP buffer from the RtpStream class to the RtpSender (this is because, once simulcast/SVC is done, we will handle different sequence numbers for each receiver).
  • Regardless how many RTP packets we store in the RtpStream instance, don't satisfy NACK requests for packets older than 200ms (thanks @lminiero).
  • Add SDES (CNAME) packet on compound packets with only Receiver Reports.
    • This is a no-sense and won't be done.
  • Forward TMMBR and TMMBN (they are dropped right now).
  • Locally generate REMB feedback messages to let remote senders adapt to the estimated bandwidth.
  • Implement draft-holmer-rmcat-transport-wide-cc-extensions (RTP Extensions for Transport-wide Congestion Control).
    • NOTE: This will be done after 1.0.0.

@ibc ibc changed the title Implement RTCP Implement more RTCP processing Feb 21, 2017
@ibc ibc changed the title Implement more RTCP processing Implement more RTCP stuff Feb 21, 2017
ibc referenced this issue Feb 27, 2017
@ibc
Change "cooming soon" text
94d2d0f
@ibc
Copy link
Member Author

ibc commented Mar 1, 2017

Note for the future: (S)AVPF spec mandates support for TMMBR which is a way to limit the sender bandwidth (REMB does also that), so we should support it.

@ibc ibc mentioned this issue Mar 2, 2017
Closed
4 tasks
@ibc
Copy link
Member Author

ibc commented Mar 9, 2017 •
edited
Loading

@jmillan one question please:

Let's imagine that mediasoup is gonna generate its own NACK and TMMBR and send them to remote senders. Which value should be placed into the SSRC of packet sender field of those feedback messages? Also, which SSRC of packet sender is currently set in Receiver Reports and Sender Reports generated by mediasoup?

Such a SSRC of packet sender is not signaled in the SDP (it may be signaled in ORTC or let the browser choose).

@jmillan
Copy link
Member

jmillan commented Mar 9, 2017

On packets generated by RtpReceiver (ie: Receiver Report), we are setting a SSRC of 0. On packets generated by RtpSender (ie: Sender Report), we are setting the SSRC value that is being sent.

This is the way to go IMHO. Instead of hacking an invented SSRC on RTCP generated by receivers just set it to 0. The receiver of such RTCP has no way to check the SSRC anyway.

@ibc
Copy link
Member Author

ibc commented Mar 9, 2017

Nice and clear.

@ibc
Copy link
Member Author

ibc commented Mar 13, 2017

Habemus NACK generation in master branch: 553f4af

@ibc ibc modified the milestone: 1.0.0 Mar 13, 2017
@ibc
Copy link
Member Author

ibc commented Mar 17, 2017

Note regarding receipt of feedback messages (in draft-ietf-rmcat-rtp-cc-feedback):

Compound RTCP packets contain a Sender Report (SR) packet and a
Source Description (SDES) packet, and an RTP Congestion Control
Feedback (RC2F) packet [I-D.dt-rmcat-feedback-message]. Non-compound
RTCP packets contain only the RC2F packet.

So we must be ready to parse feedback messages also in compound packets. But maybe this is already done, is it @jmillan? :)

@jmillan
Copy link
Member

jmillan commented Mar 17, 2017

So we must be ready to parse feedback messages also in compound packets. But maybe this is already done, is it @jmillan? :)

Absolutely!

@ibc
Copy link
Member Author

ibc commented Mar 17, 2017

Cool, so are those two "packets" implemented?

@jmillan
Copy link
Member

jmillan commented Mar 17, 2017 •
edited by ibc
Loading

No, not yet.

@ibc
Copy link
Member Author

ibc commented Mar 17, 2017 •
edited
Loading

I'm trying to figure out whether those "standard feedback messages" are already being sent by browsers or not... If so, it's up to the sender (which may be the mediasoup RtpSender) to implement any of the existing sender side bitrate estimation algorithms.

@jmillan
Copy link
Member

jmillan commented Mar 17, 2017

I'm trying to figure out whether those "standard feedback messages" are already being sent by browsers or not.

They are not being sent as far as I see.

@ibc
Copy link
Member Author

ibc commented Mar 17, 2017

So we basically can deal with draft-alvestrand-rmcat-remb (implemented by Chrome and Firefox) and draft-holmer-rmcat-transport-wide-cc-extensions (implemented by Chrome), both of them discontinued specs, right?

@jmillan
Copy link
Member

jmillan commented Mar 17, 2017

yes 👍

@ibc ibc modified the milestone: 1.0.0 Mar 28, 2017
@ibc
Copy link
Member Author

ibc commented Apr 6, 2017

And we are done here :)

@ibc ibc closed this as completed Apr 6, 2017
@ibc ibc modified the milestones: 1.0.0, v1 Nov 22, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ibc ibc

@jmillan jmillan

Labels
None yet
Milestone
v1
Development

No branches or pull requests
2 participants
@ibc @jmillan