Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch RTCP handling to pion/interceptor #22

Open
aler9 opened this issue Feb 27, 2021 · 6 comments
Open

Switch RTCP handling to pion/interceptor #22

aler9 opened this issue Feb 27, 2021 · 6 comments
Labels
enhancement New feature or request

Comments

@aler9
Copy link
Member

aler9 commented Feb 27, 2021

Features that are currently missing in gortsplib's receiver reports and sender reports, and are handled correctly in pion/interceptor:

  • support for reordered packets in receiver reports
  • NACKs
@yelodevopsi
Copy link

yelodevopsi commented May 3, 2024

Hi @aler9 ! What is the current status on this one?

We're really eager to get some stats for RTSP-sessions (accessible in Prometheus metrics or in API), especially for jitter and package loss.

Do you have any basic overview of the migration process to get this working from the webrtc package, over to to RTSP on this matter? Because if not currently in progress, I/we might try to start the process because of the need for it now. Also, if you could provide a rough overview of the needed work, this would help alot.

Some key features, that might overlap with this endeavour:

  • Would it be possible to get some features equivalent to rtpjitterbuffer (gstreamer) functionality here? E.g setting latency manually to compensate for out-of-order packages etc.

  • We're using the mediamtx repo currently, and see that the most optimal way would be to avoid using gstreamer all together because of the great cross-platform support blueenviron's go-libs have 😺

Is the gortsplib currently utilizing any optimalisation/adjustments in the videostream based on the RTCP for RTSP-sessions?

@aler9
Copy link
Member Author

aler9 commented May 3, 2024

Hello Magnus,

What is the current status on this one?

The switch to pion/interceptor was put on hold because it requires an additional layer that decreases performance. Most of its features are already implemented here.

We're really eager to get some stats for RTSP-sessions (accessible in Prometheus metrics or in API), especially for jitter and package loss

This might be a good future enhancement, currently the only two exposed stats are ServerSession.BytesReceived() and ServerSession.BytesSent(), we might add ServerSession.Stats() in order to obtain jitter and packet loss too, that are currently available inside rtpreorderer and rtcpreceiver. I've created a feature request here: #556

Would it be possible to get some features equivalent to rtpjitterbuffer (gstreamer) functionality here?

This is already present in a real-time variant, it's inside rtpreorderer. The maximum number of lost packet is fixed to 64.

Is the gortsplib currently utilizing any optimalisation/adjustments in the videostream based on the RTCP for RTSP-sessions?

Video adjustments like dynamic bitrate cannot be achieved since an RTSP server or client doesn't have access to the video encoder, contrarily to browser-based WebRTC, therefore video packets can only be routed without touching them.

Resending lost video frames through NACKs would be a very good enhancement but there is no client that supports it (VLC, FFmpeg, GStreamer), thus it would be useless to implement it.

The only thing that you can do to improve the transmission rate with RTSP is using Forward Error Compensation (FEC), that consists in sending the same packet multiple times at once. You can generate packets by following the specification and send them through gortsplib.

@yelodevopsi
Copy link

yelodevopsi commented May 6, 2024

Thank you so much for the detailed reply @aler9 ! 👏

We have a lot of unstable (wifi/wireless linked) RTSP-cameras we need to connect to.
Would it be a requirement that the RTSP-cameras support FEC, to stabilize the connection between the source and RTSP-server?

Btw, we're currently benchmarking TCP vs UDP on the streams, but are interested in all ways the gortsplib by default handles instable/bad streams, and what potential features/param adjustments we can use/create to manually improve the streams.

From what you write, I cannot find alot of FEC implementations (only some research-papers) which use this.

Thanks in advance

@yelodevopsi
Copy link

From what I've used in gstreamer, mainly three params/features would be nice to manually adjust:

  • latency (buffer, and jitter-buffer on the received source-stream): Set manually in ms
  • dropOnLatency: true/false
  • udp-buffer-size: (in bytes)

Are any if these already implemented as with any default-value?
Also, when you mention: rtpreorderer - What is the default buffer/latency here?

@aler9
Copy link
Member Author

aler9 commented Jun 22, 2024

Also, when you mention: rtpreorderer - What is the default buffer/latency here?

There is no latency, because the server doesn't need to emit frames with a constant interval. The server sends out frames to clients as soon as they are available. It's the client responsibility to keep a buffer with a latency and emit frames with a constant interval.

@tmatth
Copy link
Contributor

tmatth commented Jun 27, 2024

This might be out of scope of the original issue but is it possible currently to handle PLI RTCP feedback packets (in the context of a mediamtx server being sent them by an RTSP client)? In my case I'm needing to intercept them and forward them upstream (as mediamtx is proxying an RTSP stream from another source).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants