Add RTSP support

[tresvecesseis]

This is our first public release of our in-house RTSP Mediasource, we have to rewrite it several
times to make it useful for a wider audience and more easily maintainable and extensible.
It currently supports:

• RTSP 1.0 (RFC2326) - with Basic and Digest authentication in standard URL encoding (some
  devices use URL parameters to encode user/password info)
• SDP (RFC4566)
• MP2T over UDP
• RTP (RFC3550)
• RTP payloads:
  • MP2T
  • H264 (RFC6184) - single NAL Unit and Non-Interleaved modes
  • AAC (RFC3640) - LR and HR bitrates, simple, multiple and fragmented AAC frames
    (interleaved not implemented)
  • AC3 (RFC4184)
  • G711.1 (RFC5391) - R1, R2A, R2B and R3 modes (untested)
  • G711.0 (RFC7655)

We are working to integrate Server Trick Modes and asynchronous notifications, but we have to
clarify the best way to control a push media source (we need to modify explayerImpInternal to
communicate the player state to the media source - pause - and the media source to inform that
some events had ocurred - end of stream reached, begin of stream reached, ...) with
the Exoplayer lead developers in the meantime the trick modes have to be implemented outside the library.

[tonihei]

Thanks for the contribution. I haven't looked in detail yet, but one thing that would definitively need to be changed is that RTSP should be implemented in its own extension similar to VP9 or Opus. Please see @ojw28 comment about it here. Can you move the implementation as far as possible to its own extension?

[tresvecesseis]

We could move the RTSP implementation to an extension but we will like to maintain the RTP an SDP parts in the core library to reuse it in our RTP datasource.

[tonihei]

It's probably even better to put it in library and not extensions (similar to Dash and HLS) because it doesn't use any external third-party dependencies.

And it's probably alright to keep certain parts in the core library if they can be reused at different places.

[andrewlewis]

Regarding controlling the player and listening to player events in the media source: depending on the requirements, you may be able to use the player parameter passed to prepareSource for this. This should be fairly easy to use if you require that the source is the top-level source passed to prepare (otherwise the code in the media source interacting with the player needs to handle situations where another source in a composition is playing).

[tresvecesseis]

Another problem we have is that there is no concept of VCR like FF and RW in exoplayer (only seekTo is supported). In RTSP FF and RW is done at the server level, the player keeps decoding the frames that it receives and the server is in charge of select the I-frames (in forward or reverse order) and send them to the player, so we will need to keep these states (and their multipliers) in the library.

[googlebot]

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for the commit author(s). If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and have the pull request author add another comment and the bot will run again. If the bot doesn't comment, it means it doesn't think anything has changed.

[googlebot]

CLAs look good, thanks!

[FabianTerhorst]

Is there a possibility that this doesn´t work on the emulator. It worked great on all physical devices i have testet with, but not on the emulator. No x86 support?

[portizb]

@FabianTerhorst it doesn't work with the emulator because of a bug in android. Issue#207602: Emulator does not redirect UDP packets

[anhth58]

Hello there !
I must be display 6 thumb video rtsp in a screen. I used to try VideoView. It worked ! but there is critical problem : deylay. I pulled this pull request and atempt to use it to solve my problem. But it did not play anything without error. Any one can help me ? Thanks so much.

[portizb]

@anhth58 at the moment we are working to support trick modes and improve the design of the solution and solve the bugs that are detected. Generate thumb video rtsp isn't something we plan to support any time soon; sorry.

Anyway, could you give us more detail about what you really need? Generate a tumbail from the video keeping the rtsp live connection paused until you decide to continue playing or stopping the video?

[anhth58]

@portizb Thanks for reply. I need to play serveral video in a screen (paging) without controller, just thumb. Then click to each of them to view full screen. I have tried Exoplayer but it does not work and even without error log. Could you please tell me some guideline ? Thank you so much.

[FabianTerhorst]

I have testet this with some cameras now. For the one where it doesn´t work im getting an 200 Ok first and then an 500 Internal Server Error.

Sender: SETUP rtsp://192.168.178.28:554/12/trackID=0 RTSP/1.0
    CSeq: 3
    User-Agent: ExoPlayer/2.x (Media Player for Android v2018.02.17)
    Transport: RTP/AVP;unicast;client_port=37685-37686

Receiver: RTSP/1.0 200 OK
    Server: HiIpcam/V100R003 VodServer/1.0.0
    CSeq: 3
    Session: 179539325971263; timeout=60
    Date: Mon, 23 May 2012 06:51:26 GMT

Sender: SETUP rtsp://192.168.178.28:554/12/trackID=1 RTSP/1.0
    CSeq: 4
    User-Agent: ExoPlayer/2.x (Media Player for Android v2018.02.17)
    Transport: RTP/AVP;unicast;client_port=46994-46995

Receiver: RTSP/1.0 500 Internal Server Error
    Server: HiIpcam/V100R003 VodServer/1.0.0
    CSeq: 4

Sender: TEARDOWN rtsp://192.168.178.28:554/12/ RTSP/1.0
    CSeq: 5
    User-Agent: ExoPlayer/2.x (Media Player for Android v2018.02.17)
    Session: 179539325971263;

EventLogger: internalError [0.23, loadError]

Edit: some more logs

Sender: OPTIONS rtsp://192.168.178.28:554/12 RTSP/1.0
    CSeq: 1
    User-Agent: ExoPlayer/2.x (Media Player for Android v2018.02.17)

Receiver: RTSP/1.0 200 OK
    Server: HiIpcam/V100R003 VodServer/1.0.0
    CSeq: 1
    Public: OPTIONS, DESCRIBE, SETUP, TEARDOWN, PLAY

DESCRIBE rtsp://192.168.178.28:554/12 RTSP/1.0
    CSeq: 2
    User-Agent: ExoPlayer/2.x (Media Player for Android v2018.02.17)
    Accept: application/sdp

RTSP/1.0 200 OK
    Server: HiIpcam/V100R003 VodServer/1.0.0
    CSeq: 2
    Date: Mon, 23 May 2012 06:51:26 GMT
    Content-Base: rtsp://192.168.178.28:554/12/
    Content-Type: application/sdp
    Content-Length: 413
    
    v=0
    o=StreamingServer 3331435948 1116907222000 IN IP4 192.168.178.28
    s=\12
    c=IN IP4 0.0.0.0
    b=AS:4100
    t=0 0
    a=control:*
    m=video 0 RTP/AVP 96
    b=AS:4000
    a=framerate:15
    a=control:trackID=0
    a=rtpmap:96 H264/90000
    a=fmtp:96 packetization-mode=1;profile-level-id=420029; sprop-parameter-sets=Z00AHpWoKAv+VA==,aO48gA==
    a=framesize:96 640-360
    m=audio 0 RTP/AVP 0
    a=control:trackID=1
    a=rtpmap:0 PCMU/8000

[portizb]

@FabianTerhorst The exchange of rtsp messages between the client and the server is correct, therefore the behavior of the rtsp client is correct too. The client requests the setup for audio and video, and the server responds with error only in the audio. That is why, the client sends the teardown request.

What is the reason the server responds with error? I do not know. Maybe the server is not properly configured or the server only supports RTSP over HTTP.

Could you do the same test using a VLC client with verbose enabled (vlc -vvv)?

[floschu]

Hi.
I have built and used your implementation and for my use case it does work great. However I have a question: Is there a way to change latency or minimum buffering time? I am trying to get a near real time stream experience with a maximum of 100ms delay.

[portizb]

@floschu You can reduce the size of buffering when you create the exoplayer instance by adjusting the DefaultLoadControl parameters (bufferForPlaybackMs and bufferForPlaybackAfterRebufferMs).

[ojw28]

I took a quick look at this. Some initial comments that might assist in getting it into something closer to what we'd be happy with:

• There's a lot of code being lumped under the core util package that doesn't look like it belongs there. That package is more or less intended to be a package for things that don't belong anywhere else, but aren't substantial enough for a package of themselves. As soon as there's a sub-package in there the definition doesn't hold. If it's important enough for its own package then it shouldn't be under util. Most of util.media appears to be an Extractor implementation. If this is accurate, could it be in the extractor package instead?
• Please can you give all classes that are needed only within a package default visibility, so that they're not exposed as part of the library API. There are a large number of classes that have public visibility where it appears unnecessary, including classes that don't have any Javadoc.
• This is a pretty huge pull request. Can you provide some kind of brief documentation (e.g. a page or two) explaining why things are where they are and how they fit together. It's quite hard to know where to start when reviewing the change.
• I was wondering if would be possible to implement some (possibly lots) of this as a standalone library that ExoPlayer could depend on, rather than have it all part of ExoPlayer directly. Is it possible to split out what's ExoPlayer specific and what's general rtp/sdp, so the latter could be a separate library?

Let me know what you think. Thanks!

[tresvecesseis]

@ojw28 thanks for your review, I will try to give an answer to all your remarks. First of all, we are already rewriting/refactoring parts of the RTSP/RTP code in response to the feedback received from the testers, due to that we are introducing a reordering buffer to cope with out of order packets that seem to be a problem in some wifi scenarios, we are also improving the compatibility with a larger amount of ip webcams and adding trick modes support

We are trying to share code with our RTP extractor so we try to put common code in the util package, if that´s a problem we will move all the RTP code to an RTP extractor and make the RTSP mediasource depend on it. The SDP will become a new library that will be used by the RTSP and RTP packages.

We will fix the visibility of the classes outside the library and provide as much info as possible over the library design.

[frog1014]

Is it possible to implement RTSP over HTTP?
Thanks so much!

[warren-bank]

If I could make one feature request.. it would be compatability with being able to play rtsp streams produced by the libstreaming library.

on Android:
libVLC v2.x works well; libVLC v3.x works but (imho) not quite as well. When I tested this experimental branch of ExoPlayer about 17 months ago, it did not work (with streams specificially produced by libstreaming); I'd be glad to test it again, if somebody could tell me the best commit to use (ie: most recent/stable that compiles without error).

on Windows desktop:
VLC and ffplay both work well.

Since libstreaming is such a popular library for running rtsp servers on Android, (imho) compatability should be a priority. The library itself is incredibly compact and simple to grok.. so it's possible that it just needs a few minor tweaks (on its end) to be more standards compliant.. dunno.

[tobigun]

@googlebot I consent.

[portizb]

Hello guys! How is everything?

I feel my silence... have been hard times, but I am back!
Likewise, i need a few days to be up to date and see how the current code is.

Working in this PR is already a problem, because the code is very outdated, and also I am not the owner of the branch, although I do own the code. So, I need to work on my own branch.

My idea is to create either two new PRs, one for RTP, and one for RTSP, or just a new one with all the code. Likewise, i am open to your suggestions.

The first PR provides the RTP library and the necessary tools to play audio or video or mux (only with mp2ts over RTP). This version is an extension and improves the one used by RTSP.

The second PR provides the RTSP library (RTSP and SDP), and use the new RTP library.

Currently, I am working in two branches:

https://github.com/portizb/ExoPlayer/tree/dev-v2-rtp (The RTP code is not uploaded)

https://github.com/portizb/ExoPlayer/tree/dev-v2-rtsp

Anyway, and before doing anything, I have to take a look at the code.

Thanks for your patience and understanding.

[andrewlewis]

@portizb Welcome back and thanks for the message! @lcf87 has been working on RTSP support on our side using the latest sources as a starting point, so you may want to hold off working on this pull request further until we start publishing pieces of that (likely early next year). Our implementation is probably going to be quite minimal at first (e.g., lacking support for various media formats) so contributions will be welcome.

[portizb]

@portizb Welcome back and thanks for the message! @lcf87 has been working on RTSP support on our side using the latest sources as a starting point, so you may want to hold off working on this pull request further until we start publishing pieces of that (likely early next year). Our implementation is probably going to be quite minimal at first (e.g., lacking support for various media formats) so contributions will be welcome.

@andrewlewis Excellent! So, I will wait until that day and then, I will take your implementation as a starting point to continue contributing.

[portizb]

@googlebot I consent

[google-cla]

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

[lars18th]

Hi @tresvecesseis ,

I only want to ask if this RTSP support will be included at some point inside the master branch of the ExoPlayer.

Thank you for your effort.

[andrewlewis]

@lars18th My previous comment still applies. We expect to have a minimal, official implementation to share in the near future. Thanks for your patience!

[Neustradamus]

Any news on this PR?

[Pitel]

Can you also add support for RTSPS (RTSP over SSL). Nest cameras uses it. Or is there some trick to make it work out of the box?

[lars18th]

Hi,

I suggest to add simple SAT>IP compatibility to this source. One relevant change is the requirement to understand the "com.ses.streamID" parameter of the SETUP command as the session ID. See this inside the VideoLAN source:
https://github.com/videolan/vlc/blob/e3f38be0ccab13d46041de208bcceb917278f289/modules/access/satip.c#L322

[lcf87]

ExoPlayer release 2.14 included our RTSP support implementations! It supports playback with RTSP/1.0 (RFC2326) with RTP/AVP (RFC1889). Please reference our more in-depth info here.

Format support:

• H264 (RFC3984, Single/STAP/FU mode)
• AAC with ADTS (RFC3640, single/fragmented mode, LBR/HBR, multi-channel)
• AC3 (RFC4184)

Network support:

An RTSP server sending media data carrying RTP packets over UDP is (inherently) connectionless and will not pass through a NAT. Streaming with TCP is required behind a NAT.

We support streaming unicast RTP with UDP and TCP (also known as RTP-over-RTSP). In the current implementation, if no RTP packet is received on UDP, ExoPlayer will retry playback using TCP.

To try it out

Simply add an entry to the demo app's media list here

For instance:

 {
    "name": "Wowza Amazon EC2 - Big Buck Bunny Comedy Film",
    "uri": "rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_175k.mov"
  }

Debugging tip

• To see received RTSP responses, add log here
  

  ExoPlayer/library/rtsp/src/main/java/com/google/android/exoplayer2/source/rtsp/RtspMessageChannel.java

  Line 288 in 92e05bc

  messageListenerHandler.post(

• To see outgoing RTSP requests, add log here
  

  ExoPlayer/library/rtsp/src/main/java/com/google/android/exoplayer2/source/rtsp/RtspMessageChannel.java

  Line 210 in 92e05bc

  public void send(List<String> message) {

[sproctor]

Is it possible to play streams that require authentication? If so, could you give an example?

[notjulian]

Is it possible to play streams that require authentication? If so, could you give an example?

I think the easiest way it is

rtsp://user:password@ip:port

[lars18th]

I suggest to add simple SAT>IP compatibility to this source. One relevant change is the requirement to understand the "com.ses.streamID" parameter of the SETUP command as the session ID. See this inside the VideoLAN source:
https://github.com/videolan/vlc/blob/e3f38be0ccab13d46041de208bcceb917278f289/modules/access/satip.c#L322

Hi, It's possible that someone address the compatibility with SAT>IP servers, please?

[Pitel]

@sproctor @notjulian Basic Auth works with this MR's source. But it doesn't work with the preliminary RTSP support in 2.14.0. See #8941.

[sproctor]

@Pitel Thanks. My source uses digest authentication, which is supported on https://github.com/portizb/ExoPlayer/tree/dev-v2-rtsp . I hope it gets support here too.

[warren-bank]

Has anyone else tried playing RTSP with r2.14.0 and experienced this issue?

Unless I'm completely missing a step, that is what I see when I try to play:
rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_175k.mov