New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: Live view proxy #23

Open
anthonyangel opened this Issue Mar 13, 2017 · 55 comments

Comments

@anthonyangel

anthonyangel commented Mar 13, 2017

Hi, would it be possible to use this library to allow a continuous live view stream to be exposed locally as an IP camera feed (either ONVIF or just an mp4 stream), so that it could be recorded by an NVR?

Ideally this would bypass Rings servers and just stay within the LAN (would reduce the overheads in terms of data transmission), but that doesn't seem possible at the moment, but this would allow Ring to be integrated into an existing CCTV installation instead of existing as a separate system.

Thanks,
Anthony

@tchellomello

This comment has been minimized.

Owner

tchellomello commented Mar 14, 2017

@anthonyangel if you call the function live_streaming_json you will get a JSON output with the data to connect to the SIP streaming. Maybe we could use this function to implement this proxy, however not sure if that would lock the device for other notifications.

@anthonyangel

This comment has been minimized.

anthonyangel commented Mar 17, 2017

Thanks, I'd had a look at that already for 1-off uses it works well. The initial issues I can foresee are:

  • Timeout on the live stream, I think that the URL needs to be refreshed every 10 minutes
  • Whether having this connected locks out the device
@Klathmon

This comment has been minimized.

Klathmon commented Apr 22, 2017

I'm looking into this a bit more. I'm able to grab the video and audio using an SIP program (interestingly enough the stream only works without encryption... so i'm guessing that these videos are being sent across the net entirely un-encrypted...)

The stream returned from live_streaming_json expires in about 3 minutes for me. It seems that the expires_in field is in seconds, and it varies, requiring you to grab a new stream when it expires and maybe stitch them together at some point?

The doorbell still works while it's streaming, however i'm not sure if notifications will still go off on the phone when you are using it. I'll have to see if I can test this at another point, but I have a feeling it will work the same as I have been watching the stream on my phone (through the official app) while my wife tests it and it rang her phone, so I'm assuming it will work correctly.

I'm not the most versed in python, but there does seem to be a library for SIP in python that could be possibly used here. With that it might be possible to grab a raw video stream out.

So TL;DR: it does look like this is possible, but it's not going to be all that easy, and it's probably way outside my ability in python.

@jlippold

This comment has been minimized.

Contributor

jlippold commented Apr 23, 2017

@Klathmon What SIP client did you use? There's are some sip libraries that support WebRTC with websockets but I'm shooting in the dark because I can't even get a simple player going.

@Klathmon

This comment has been minimized.

Klathmon commented Apr 23, 2017

I used the Blink client to do my testing. I was also looking at the WebRTC players that interface with SIP but decided to hold off as they wouldn't integrate as well with a python project like this (or home assistant).

By calling the SIP line I was able to get 2-way audio working instantly, but in order to get video working I had to go into Blink's Preferences and under "Accounts", switch the tab to "Media" and deselect "Encrypt audio and video" under "RTP Options"(i also unchecked "Send inband DTMF" but I don't think that applies here).

FYI if you have neighbors, the second you connect you will be broadcasting your computer's microphone to the ring speakers until you hang up! I think I scared the hell out of mine with a very loud "Holy shit it works!" the first time it connected!

@tchellomello

This comment has been minimized.

Owner

tchellomello commented Apr 23, 2017

That is very cool. Would be awesome to have this working in home assistant.

@jlippold

This comment has been minimized.

Contributor

jlippold commented Apr 23, 2017

That blink SIP client is expensive, let me know if you get video working, if you do then it's worth looking into an open source python lib

@Klathmon

This comment has been minimized.

Klathmon commented Apr 23, 2017

@jlippold I'm using the free/trial version and it is working well enough for now. It was also the first one I tried so others might work.

And I forgot to finish that sentence, it works with video if you uncheck using encryption. Both video and audio work great.

@jlippold

This comment has been minimized.

Contributor

jlippold commented Apr 23, 2017

I assume you're on windows maybe? the mac app isn't free

@Klathmon

This comment has been minimized.

Klathmon commented Apr 23, 2017

Yes, windows.

It does seem to be bone-stock SIP so any client that let's you disable encryption should work for both audio and video.

@jlippold

This comment has been minimized.

Contributor

jlippold commented Apr 23, 2017

yea man, this works well.. For anyone wondering the sip address, is the sip_to property on the json. No other config needed, just that address. The SIP address expires after 10 seconds or so

@haimiko

This comment has been minimized.

haimiko commented Apr 24, 2017

In the sip_to, is the is username-password@sipurl:port ?

@jlippold

This comment has been minimized.

Contributor

jlippold commented Apr 24, 2017

There's no password needed, just that sip_to property value... The "security" is that the sip to address expires frequently. FWIW, I only got it working with the windows version of that blink software/

@haimiko

This comment has been minimized.

haimiko commented Apr 24, 2017

@jlippold

This comment has been minimized.

Contributor

jlippold commented Apr 24, 2017

if you can dynamically pull the sip recording, let us know. I spent a whole day trying and gave up

@Klathmon

This comment has been minimized.

Klathmon commented Apr 24, 2017

You should be able to use any client that allows "Direct SIP dialing".

So when you setup a new SIP app, it will ask you for server, username, password, etc... You don't need any of that. You just need to be able to dial an SIP URI directly.

Other than that, you need to make sure you are using TCP not UDP (might be a setting in the options), and you need to ensure that you are not using any encryption (might be under a setting labeled RTP or something similar)

I just tried it with an android app MizuDroid and got it to work once, but you only have like 10 seconds from when you get the address till when you can't dial it any more.

I'm still looking into how the SIP protocol works and how to grab the RTP stream directly and mess with it. I'm confident we can get this working.

@Klathmon

This comment has been minimized.

Klathmon commented Apr 24, 2017

Okay, i'm finally beginning to get this figured out.

So I've been doing some reading up on SIP (RFC 3261 and RFC 6665 are the big ones used).

I also managed to capture a full session using wireshark while i was testing with the Blink SIP client (don't want to share that publicly as i'm pretty sure the ring exposes enough info to let anyone connect to my camera in this communication).

A few things i've found out:

SIP is just an "initiation" protocol, all it does is setup another protocol which actually streams the data. that protocol seems to be RTP (which from a preliminary look, should be easy to convert to just about any video format). The SIP messaging responds with information on how to access this RTP stream, and it seems like it's randomized somewhat so we can't just skip the whole SIP messaging step and grab the stream.

It also looks like the version of SIP here uses some extensions (the second RFC I linked) to "subscribe" to something. That subscription request is being denied which is why i'm guessing these streams always end like 30 seconds after I start them.

I'm fairly sure we can get the SUBSCRIBE stuff to work by just impersonating the ring client, which means we can get a full stream (including 2-way audio if you want it) working.

All that being said, I'm gonna give this a shot. I haven't worked with python for like 7 years, so i'm hoping if I can get a "proof-of-concept" working, someone else can take the reins and actually get it working well.

My next step is to get some of this SIP messaging working in python. I'm going to use the Twisted library which does seem to have some support for SIP (here's the protocol definition in twisted).

@haimiko

This comment has been minimized.

haimiko commented Apr 24, 2017

@Klathmon

This comment has been minimized.

Klathmon commented Apr 24, 2017

Sorry to say but I think i'm throwing in the towel here (at least for a while).

I just don't know python well enough to be able to get anywhere, and i really don't think it will be very beneficial to this project if I spent the time to implement it in another language...

Here's some stuff I found, hopefully it's useful for someone to pick up:

  • p2p-sip seems to have a pure-python sip implementation in it. at the very least it's inspiration.
  • SIPPing a SIP packet forging tool. This was what I got the furthest with, but still can't get a full send->response working even for the initial "INVITE" packet.
  • Twisted's SIP class. It does seem like this will work great, but I spent like an hour trying to figure out how to send the request before giving up.
  • Blink the SIP client i've been using to do all of the exploration and testing. As long as you disable encryption in the settings as described further up in the thread, it works great. Make sure to make the request within like 30 seconds of calling live_streaming_json

Also, wireshark has proven to be super helpful in tracking down what is being requested and the responses.

I might pick this up in a week or so, but as of right now i'm just spinning and not getting anywhere.

@tchellomello

This comment has been minimized.

Owner

tchellomello commented Apr 25, 2017

@Klathmon you helped a lot already. Thank you so much for studying the protocol.

I want to get back to this RFE too which will be awesome to have this support.

Thanks everyone involved and let's try to get this working. I'll try to look at the libraries you pointed out too.

@Klathmon

This comment has been minimized.

Klathmon commented Apr 25, 2017

If you or anyone else are able to get the python plumbing started, I can muddle my way through the actual implementation of the back-and-forth and grabbing the stream data.

I actually love reverse engineering like that, it's just trying to do this in a language I haven't used for years is biting off more than I can chew.

@IanMitchell77

This comment has been minimized.

IanMitchell77 commented May 30, 2017

I can get the alert and decode the JSON string but what did you put in the Blink SIP client to get the video? sip_to?

Many thanks in advance!

@Klathmon

This comment has been minimized.

Klathmon commented May 30, 2017

I believe it's the sip_to field (going from memory here). You do need to make sure you do this to get video working in the Blink SIP client:

go into Blink's Preferences and under "Accounts", switch the tab to "Media" and deselect "Encrypt audio and video" under "RTP Options"

Also you gotta be fast, once you get the JSON back you have like 30 seconds to get the URL and open the video feed before it expires.

@IanMitchell77

This comment has been minimized.

IanMitchell77 commented May 30, 2017

Thanks!

Did you find a Linux cli or gui that worked?

@IanMitchell77

This comment has been minimized.

IanMitchell77 commented May 30, 2017

I can get AV with the Blink software!

However I only get a few seconds before it disconnects - does anyone have any idea on what I need to do to make it last longer?

@jlippold

This comment has been minimized.

Contributor

jlippold commented May 31, 2017

you have to requery the endpoint everytime it expires to get a new sip address

@IanMitchell77

This comment has been minimized.

IanMitchell77 commented May 31, 2017

Maybe it is that, I'll check - Thanks!

It's seems that the sip token in the JSON response may be relevant here but I'm not experienced in SIP and don't have an idea what to do with it.

It takes me about 5 seconds to put the sip_to string in blink, I then call and video lasts for a few seconds. I thought it expires much later than that but has anyone automated getting the sip_to string in to blink and getting a new string when it finishes?

@rpavez

This comment has been minimized.

rpavez commented Jun 8, 2017

I haven't I might try next weekend.

@jlippold

This comment has been minimized.

Contributor

jlippold commented Jun 9, 2017

I tried. I don't think ring uses websockets, and jssip does

@rpavez

This comment has been minimized.

rpavez commented Jun 9, 2017

Another good option for fine control would be a command line sip client, here are some useful links I found.

From Stackoverflow: SIP-Client for Raspberry Pi that works from command line?

  1. Linphone: Easy to install but I fail to make phone calls
    RaspberryPI: Making SIP outbound calls using linphonec or an alternative SIP soft phone
  2. Ring (formerly SFLphone): Looks promising but needs to be installed from source
  3. PJSIP (C Library) See http://www.pjsip.org
  4. Twinkle CLI
  5. https://stackoverflow.com/questions/15918996/looking-for-a-sip-client-that-can-use-an-h-264-rtsp-stream-as-a-video-source / openRTSP / playSIP
  6. https://github.com/AGProjects/python-sipsimple
  7. https://github.com/alfredh/baresip / https://github.com/alfredh/baresip/wiki/Using-Baresip:-Basic-Commands
  8. https://www.npmjs.com/package/sip-simple
  9. https://github.com/ecelis/sck

Getting some ideas from here I feel it should be possible manually negotiation SIP to get the source of the streaming and then use ffmpeg to transcode/stream.

@IanMitchell77

This comment has been minimized.

IanMitchell77 commented Jun 9, 2017

I tried the jssip website and that does not work.

Has anyone had any joy with the Blink software on Linux? I'm using Ubuntu 14.04 but can switch easily to make life easier, its only a VM.

@rpavez

This comment has been minimized.

rpavez commented Jun 9, 2017

Found this article about building Blink from the source code for OSX https://gist.github.com/lucaspiller/8194862 maybe we can study how is implemented and extract necessary modules.

@rpavez

This comment has been minimized.

rpavez commented Jun 12, 2017

Using wireshark and osx ring doorbell client I was able to capture the SIP call flow, it may help someone trying to implement a custom sip connecting figure out the process.
screen shot 2017-06-12 at 12 27 37 am

@cryptocake

This comment has been minimized.

cryptocake commented Jun 23, 2017

Has anyone tried to connect directly to the Ring doorbell? I port scanned mine and it showed 1 opened port: 1523. I'm trying to see if I can get any response from it RTP / SIP.

@IanMitchell77

This comment has been minimized.

IanMitchell77 commented Jul 26, 2017

Any joy cryptocake?

@joeyberkovitz

This comment has been minimized.

joeyberkovitz commented Aug 13, 2017

Artik cloud (https://artik.cloud/works-with/ring.html) gives access to the SIP URI, but indicates that there's no TLS on the one that it provides. Any chance we can connect through that?

@asantaga

This comment has been minimized.

asantaga commented Aug 16, 2017

@Klathmon can u give step by step instructions how to get the video streaming working from within Blink?

Ive followed what you have above and get no connection.. perhaps Im going mad.. or things have changed..

@asantaga

This comment has been minimized.

asantaga commented Aug 17, 2017

As an interesting comment, I tried artik cloud (above from Joeyberkovitz) and whilst it did return a RTSP URL, the domainname wouldnt respond... very strange..

@joeyberkovitz

This comment has been minimized.

joeyberkovitz commented Aug 21, 2017

I was able to play with the sip_to URI from artik cloud and figured out that it accepts connections if the user agent matches the Blink Windows client.
I attached a basic call script and config file that can connect to the doorbell and then conference the call to another URI. It's not perfect, but it definitely works. To get around the connection ending, it should be possible to get a new URI and add it to the conference when the first one ends.

config.txt
conferenceCall.py.txt

It looks like the RTSPS stream works, but I had a hard time finding a compatible client

@Sfinx

This comment has been minimized.

Sfinx commented Sep 3, 2017

To be able to get the video stream always you must send the custom INFO header just after ACK (connect) from their side. The INFO packet must look like:

....
INFO sip:.....
Via: SIP/2.0/TLS ....
From: <sip:....@ring.com>;tag=xxxx
To: "FS Doorbot" <sip:.....
CSeq: 21 INFO
Call-ID: ...
Max-Forwards: 70
Content-Length: 24
Content-Type: application/dtmf-relay
X-Ding: .... id from json...
X-Authorization: 
....

Signal=2
Duration=250

And just send BYE to close the SIP session to the doorbell

@jlippold

This comment has been minimized.

Contributor

jlippold commented Sep 5, 2017

It seems like we have all the details, but we dont have anyone who can write a SIP client in python

@Klathmon

This comment has been minimized.

Klathmon commented Sep 5, 2017

So I kind of abandoned this issue for a while, but I have done a bit more research..

I don't think Ring is going to let us stream from their servers for any extended amount of time. In my testing with some stuff "outside" of the things being explored in this issue, I found that even the actual apps from Ring will disable video for a while if you try to keep it streaming for many hours. Not to mention that they save all the video on their servers if you have the paid plan, and I'm almost 100% positive they will not be too happy and may start closing accounts if we fill their servers with tons of video data.

So while I think the SIP will work, we would probably want to limit it to the few minutes that the app allows, and go out of our way when programming our client to not abuse it and get the attention of Ring themselves.

That being said, if there is a way to directly talk to the ring device without touching their servers, that would be a much better solution. I'm going to be temporarily moving into a place soon where I won't be able to use the ring for a few months, so I am planning on wiring it up at my desk and doing a ton of packet-capture and seeing what it will accept/reject. Hopefully that will give us a better way of accessing this stuff that's much less likely to be shut down.

@jer78

This comment has been minimized.

jer78 commented Oct 2, 2017

Personally I’d just like a feature that captures the last motion image and updates Home Assistant. If tapping on the image gets the live view, that would be cool too, or even just play the last motion recorded is fine for my purposes.

My ultimate request would be for Home Assistant to send the video to my TV as a PIP video feed when there’s motion or a doorbell ring. But I think that’s a ways off.

Thanks for everyone’s research into this. I’d happily code if I knew where to start but this is beyond my ability at the moment.

Cheers!

@tchellomello

This comment has been minimized.

Owner

tchellomello commented Oct 5, 2017

@jer78 Please take a look at https://community.home-assistant.io/t/ring-doorbell/7943/237

I'm testing a prototype to playback the video on HASS. It would be awesome if you test it.

@jlippold

This comment has been minimized.

Contributor

jlippold commented Oct 5, 2017

@tchellomello your gist is for recorded videos, not live viewing correct?

@tchellomello

This comment has been minimized.

Owner

tchellomello commented Oct 5, 2017

@jlippold correct!! We are not there yet for the live videos...

@untotren

This comment has been minimized.

untotren commented Oct 24, 2017

@Klathmon live video api has been figured out already and documented:

http://www.kidder.io/2017/07/04/ring-doorbell-api/

@untotren

This comment has been minimized.

untotren commented Feb 17, 2018

@Klathmon were you able to make any progress on live video with the documented API?

http://www.kidder.io/2017/07/04/ring-doorbell-api/

@Klathmon

This comment has been minimized.

Klathmon commented Mar 22, 2018

@untotren I haven't tried it yet. I'm in the process of moving and ended up accidentally shipping my ring to be held at storage at my new location with a lot of other stuff so I haven't had it to play with for a while.

Come April/May I may have more time to really dive into this once we are in the new house and settled in!

@egeste

This comment has been minimized.

egeste commented Mar 26, 2018

I just read through the doc. It looks like what's happening is:

  1. The client application needs to send a request to the ring API to indicate that a user is awaiting a live video feed.
  2. At some point within 17 seconds, the doorbell will poll the ring API to see if anyone requested a live video feed, thus returning a 200 OK response.
  3. Upon the 200 OK response, the doorbell will perform a POST operation against the ring API to retrieve the necessary configuration to initiate an SIP connection. The response body of this request is noted as being unidentified by the author.
  4. Upon receiving the necessary SIP configuration, the doorbell will connect to the SIP call and start streaming video.

It's interesting to note that the author of that article did not investigate the client-side behavior of the ring app. Further investigative work needs to be done to identify:

  1. How do I initiate a request for a video on demand?
    • Can be discovered by reverse engineering the app/sniffing traffic.
  2. What kind of response does the API return once I've requested a video connection?
    • Probably returns some kind of encoded SIP connection information.
  3. How do I connect to the video stream?
    • Again, probably SIP.

Edit: I just read #23 (comment)

I 100% agree here. Streaming SIP through ring's servers is a terrible idea. One potential solution might be something like a MITM proxy? Here's what I'm thinking:

  1. Run your own SIP server, or include a small one in python-ring-doorbell.
  2. Intercept all calls to GET /doorbots_api/vod/ready so that they always return 200 OK.
  3. Intercept all calls to POST /doorbots_api/vod so that it always returns a configuration object pointing to your internal SIP server.
@GreenZapdos

This comment has been minimized.

GreenZapdos commented Apr 10, 2018

Has anyone been able to connect to the sip stream recently? Can you tell us what client you used to view the stream? It looks like the doorbell.live_streaming_json isn't working right now. doorbell._ring.query(("https://api.ring.com/clients_api/dings/active?api_version=9&burst=1"), method='GET', raw=True) works though and I was able to get a response from that. I put the value from sip_to into Linphone but wasn't able to get a connection. I'm not sure if I am using the client right though. There is a sip_token value that I think could be required to make the connection.

@HipsterZipster

This comment has been minimized.

HipsterZipster commented Oct 5, 2018

What's the current status on this issue? I'm interested in writing something for a Synology, javascript preferably

@jonFisher99

This comment has been minimized.

jonFisher99 commented Nov 12, 2018

I've avidly followed all the strategies discussed on here. It seems using the Artik Cloud to capture events and provide the SIP and RTSP URI's will be the optimal route to getting the live video feed. However, I'm finding A) The RTSP URI provided via Artik just doesn't work at all B) The SIP_TO parameter can be used with Blink SIP client, however only the audio NOT the video feed comes through, even if you remove the encryption (as suggested). Has anyone got a C# SIP client code example (perhaps using Ozeki?) to view the SIP_TO feed, or even just a SIP client software that will actually display the video feed?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment