Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document the Streaming API #97

Open
timdorr opened this issue Dec 11, 2018 · 50 comments
Open

Document the Streaming API #97

timdorr opened this issue Dec 11, 2018 · 50 comments
Labels

Comments

@timdorr
Copy link
Owner

@timdorr timdorr commented Dec 11, 2018

While there is support in the Ruby code for the Streaming API, the docs are completely empty. We should fix that by filling them in. Anyone want to help?

@jonasman
Copy link
Contributor

@jonasman jonasman commented Mar 23, 2019

Streaming seems to not work on that endpoint anymore. Even for old cars.
Used my iOS lib and got this:

Opening Stream to: https://streaming.vn.teslamotors.com/stream/XX/?values=speed,odometer,soc,elevation,est_heading,est_lat,est_lng,power,shift_state,range,est_range,heading
Stream open
Stream data: <html>
Stream data: <head><title>502 Bad Gateway</title></head>
Stream data: <body bgcolor="white">
Stream data: <center><h1>502 Bad Gateway</h1></center>
Stream data: <hr><center>nginx</center>
Stream data: </body>
Stream data: </html>
Stream error: nil```
@davidhodge
Copy link

@davidhodge davidhodge commented May 5, 2019

Having the same issue as @jonasman

@jonasman
Copy link
Contributor

@jonasman jonasman commented Jul 8, 2019

Seems like i can get a few values out of the streaming before the 502 starts.

@karekaa
Copy link
Contributor

@karekaa karekaa commented Jul 19, 2019

I see one possible thing to test out here:
https://www.lifewire.com/502-bad-gateway-error-explained-2622939

«Clear your browser cache [and/or cookies]. Outdated or corrupted files that are being stored by your browser could be causing 502 Bad Gateway issues ... »

@MrgSub
Copy link

@MrgSub MrgSub commented Sep 20, 2019

I tried the streaming API recently and it still doesn't work.

@timdorr
Copy link
Owner Author

@timdorr timdorr commented Sep 25, 2019

The HTTP streaming endpoint is completely dead. It 502's for my car, which was still supported as of last year.

I've updated the gem to use the new WebSocket endpoint: 353cfc5

@TheSpaceCowboy
Copy link

@TheSpaceCowboy TheSpaceCowboy commented Oct 9, 2019

My experience is that the 502's began in May/19 and have persisted. At that point I switched to a web-socket streaming implementation by dirkvm ( https://github.com/dirkvm/teslams ) forked from teslams. That worked until late summer at which time I began getting "data:error" messages indicating "vehicle_disconnected". At that point the code close the socket and retires with no success. The complete message from the data:subscribe attempt is:
data:error { msg_type: 'data:error', tag: 'XXX my vid XXX', value: 'disconnected', error_type: 'vehicle_disconnected' }
Are there any current streaming endpoint descriptions available, that are known to work?

@timdorr
Copy link
Owner Author

@timdorr timdorr commented Oct 9, 2019

Mine works. It's actively running as I write this, for a project I'm working on right now.

@TheSpaceCowboy
Copy link

@TheSpaceCowboy TheSpaceCowboy commented Oct 9, 2019

Will look into integrating into my processes. Thanks.

@jonasman
Copy link
Contributor

@jonasman jonasman commented Oct 20, 2019

I will also implement this on TeslaSwift lib as soon as i get some time. Will report if it works. The HTTPSteamEvent stream is totally dead even for older cars.

@andrewgreenwood
Copy link

@andrewgreenwood andrewgreenwood commented Nov 5, 2019

A couple of things to add to this:

You can send a "data:subscribe_oauth" msg_type instead of "data:subscribe". This uses the oauth token string instead of the token returned with the vehicle details.

If you respond to the "data:error" with a "data:subscribe_oauth", it seems to continue sending data so it does not look like closing and reopening the socket is necessary. Probably works the same with "data:subscribe" but I have not tested that.

EDIT: On further testing this does not seem to be as reliable as I had at first thought. And it seems that the Tesla app also performs a disconnect/reconnect if it receives a "data:error" as well.

My guess is that there's probably some message that needs to be sent in response to the "data:update" to indicate that further updates are wanted. Either that or the vehicle sends one update and then disconnects if it is parked and locked? (Would make sense - I've not yet tried streaming any data and had the vehicle in motion but I'll give that a try at some point.)

EDIT: The app does not seem to send anything in response to the "data:update", though I may have missed something. Also there are disconnections even if the vehicle is moving, but that may be due to it switching from WiFi to cell data (I'll test further and update if I find anything else.)

@ddaddy
Copy link

@ddaddy ddaddy commented Nov 18, 2019

I've been playing with this today and i've come to the conclusion that data:error vehicle_disconnected is a time out issue, albeit a very small timeout window.

If I connect to the stream while my car is parked, I get 1 result with data, then a couple of seconds later I get the data:error.

If i'm driving when I connect, it streams fine.

If I come to a stop, but have my heating on, it streams fine as the power(kw) keeps changing between 1 & 2 which pushes data to the stream.

If I then turn off my heater, so none of the data from the stream is changing (Same location, same speed 0, same power etc..) then a few seconds later it disconnects with the error.

I can't yet confirm if the timeout is coming from Tesla or from my code. The web socket connection is greeted with {"msg_type":"control:hello","connection_timeout":0} so I wonder if there is a control command we can send to extend the timeout, or else an alternative command we can send to keep the socket alive before timeout.

EDIT----

Timeout seems to be 10s with no data
I've tried sending all sorts of comment to ping or extend the timeout but nothing seems to give a response.

@timdorr
Copy link
Owner Author

@timdorr timdorr commented Nov 19, 2019

There is a control:ping on the Summon API. You might try that.

@ddaddy
Copy link

@ddaddy ddaddy commented Nov 19, 2019

Yes I tried that amongst other things. I tried sending the ping every second and although it didn’t return any error, it still timed out after 10s.
The connection to the socket doesn’t close, just the subscription to vehicle data.

What works is simply sending the subscribe command again after you receive vehicle_disconnected.

@TheSpaceCowboy
Copy link

@TheSpaceCowboy TheSpaceCowboy commented Nov 19, 2019

@andrewgreenwood
Copy link

@andrewgreenwood andrewgreenwood commented Nov 25, 2019

My partner is borrowing the car today for a short journey so I had an opportunity to observe the data whilst the car was being driven.

Sometimes after a vehicle_disconnected message, even subscribing again does not seem to work, so my previous suggestion might not be useful after all.

Observations:

Generally you'll get 1 or 2 messages if the car is parked and awake before it sends the vehicle_disconnected message. Certain things trigger more frequent messages to be sent (the car being in motion is the obvious one, but simply having the driver-side door open seemed to result in more frequent messages being sent.)

While the car was being driven, I'd get multiple messages per second. The car stopped at a red light and this resulted in a vehicle_disconnected after I received a message indicating its speed being 0. My script sent several subscribe messages 10 seconds apart, and after a minute gave up and reconnected. It then did not receive any messages after attempting to subscribe (on the new connection), and again sent several subscribe messages 10 seconds apart, giving up after a minute. The third connection attempt was successful and I started getting messages, however the car was already moving and had made significant progress along the road at that point.

I suspect if I had just disconnected and reconnected immediately each time upon receiving the vehicle_disconnected message, this would've worked.

@longzheng
Copy link

@longzheng longzheng commented Oct 14, 2020

I want to confirm if anyone else is seeing this behaviour when using the streaming API in a third-party app like Teslamate while also using the (official) Tesla mobile app.

It appears the streaming API only supports one client at a time, which the Tesla mobile app also uses. While a third-party app is using the streaming API, the Tesla mobile app seems to have issues with the location display while the car is in motion. Frequently it would error (I assume it doesn't handle error responses well) and display the car's location in a random location on the map like in the middle of the ocean or in another continent.

For example while my car was in Melbourne, Australia, it is displayed as in the middle of the Pacific Ocean

If this is a widespread issue, I think it is worth adding some warnings to this API that it may cause issues with the Tesla mobile app.

@scadaguru
Copy link

@scadaguru scadaguru commented Oct 14, 2020

I want to confirm if anyone else is seeing this behaviour when using the streaming API in a third-party app like Teslamate while also using the (official) Tesla mobile app.

It appears the streaming API only supports one client at a time, which the Tesla mobile app also uses. While a third-party app is using the streaming API, the Tesla mobile app seems to have issues with the location display while the car is in motion. Frequently it would error (I assume it doesn't handle error responses well) and display the car's location in a random location on the map like in the middle of the ocean or in another continent.

For example while my car was in Melbourne, Australia, it is displayed as in the middle of the Pacific Ocean

If this is a widespread issue, I think it is worth adding some warnings to this API that it may cause issues with the Tesla mobile app.

I have noticed same issue and my Android phone doesn't show real time update when my family member is driving Tesla Model 3. I am using TeslaMate which uses real time streaming I guess and that shows real time update on my computer/browser but not Tesla App on my Android phone.

@ddaddy
Copy link

@ddaddy ddaddy commented Oct 20, 2020

I was doing something with the streaming API the other day, and half way through the day I stopped getting any data returned from the car.
I've not been able to get any streaming data since.

I receive the

{"msg_type":"control:hello","connection_timeout":0}

handshake and send the subscribe command

{"msg_type":"data:subscribe","tag":"1305444632","token":"Base64(<email:1stToken>)","value":"speed,soc,power"}

but get absolutely nothing until it times out.
I've tried sitting in the car and messing with the AirCon, driving forwards and backwards. Nothing.

Anyone else seen this before?

@jonasman
Copy link
Contributor

@jonasman jonasman commented Oct 20, 2020

@ddaddy
Copy link

@ddaddy ddaddy commented Oct 20, 2020

Thanks.
The Tesla app seems to work fine with the correct location. I just get absolutely nothing from the socket without any changes.
I haven't had an update for a while, so hoping things aren't just stuck in the car.

@ktoonsez
Copy link

@ktoonsez ktoonsez commented Nov 11, 2020

I keep seeing so many conflicting stories on this subject. Is this something that is still working, any links to something working for android or Windows (I am clueless on ruby)?

@coleged
Copy link

@coleged coleged commented Nov 11, 2020

Despite combing through others code in languages/dialects I have no experience with, but confident I've implemented the same interface (in C++) Ive seen nothing except endless {"msg_type":"control:hello","connection_timeout":0} on the streaming websocket. And just today I'm now getting a 408 on the (REST) wake_up endpoint unless I first poke the API with the Tesla app, whereas It was working a week ago

@ktoonsez
Copy link

@ktoonsez ktoonsez commented Nov 11, 2020

@coleged, I got some code going today but looks like the wss stopped working due to tesla changing something or some kind of problem. I am not the only one getting the error from what I am reading, which is "Basic Auth Disabled". You get it right after you send the "subscribe" request.

@ddaddy
Copy link

@ddaddy ddaddy commented Nov 12, 2020

They must have shut down the basic auth. Send the oauth header instead #97 (comment)

@timdorr
Copy link
Owner Author

@timdorr timdorr commented Nov 12, 2020

Yeah, that makes sense with the new SSO system going live (must have been what took MFA so long). I'll get the Ruby gem updated soon.

@timdorr
Copy link
Owner Author

@timdorr timdorr commented Nov 12, 2020

Yep, I can confirm you need a new connect message. Here's what we generate in Ruby:

    def streaming_connect_message
      {
        msg_type: 'data:subscribe_oauth',
        token: client.access_token,
        value: 'speed,odometer,soc,elevation,est_heading,est_lat,est_lng,power,shift_state,range,est_range,heading',
        tag: self['vehicle_id'].to_s,
      }
    end

So, just send the msg_type: 'data:subscribe_oauth' and the token set to the Oauth access token you use for the Owner API. Pretty simple change, luckily.

Now I wonder what they're doing with the tokens from the vehicle data... 🤔

@ktoonsez
Copy link

@ktoonsez ktoonsez commented Nov 12, 2020

@timdorr, How do I get the "Oauth access token you use for the Owner API"? I cannot seem to find a way to do this. Thanks

@ktoonsez
Copy link

@ktoonsez ktoonsez commented Nov 12, 2020

Lol, I swear I tried that yesterday using my vehicle token. I will try again.

@ktoonsez
Copy link

@ktoonsez ktoonsez commented Nov 12, 2020

Wow, now I am getting "Got HTTP status: 403 Forbidden for . Response: {"response":"unauthorized"}`","error_type":"client_error"" with a valid bearer token. Im about to give up.

@mseminatore
Copy link

@mseminatore mseminatore commented Nov 12, 2020

@timdorr I've updated my code to use the new connect message format but an receiving an Error: Can't validate token response. Any ideas?

@timdorr
Copy link
Owner Author

@timdorr timdorr commented Nov 12, 2020

You shouldn't be using one of the two tokens entries from the vehicles endpoint. Make sure you're using the same token as the Bearer token used on the Owner API for all endpoints.

@ktoonsez
Copy link

@ktoonsez ktoonsez commented Nov 12, 2020

@timdorr, thanks for posting the code up, finally figured out what I was doing wrong. I used the "id" from the vehicle list like you do on the /vehicle API instead of using the other "vehicle_id" and it now works. Not sure if it is as expected but I do get the following and maybe you could tell me if that is what happens?

**This is with the car parked and awake.

  1. Connect to the socket and get "{"msg_type":"control:hello","connection_timeout":0}"
  2. Send the "subscribe_oauth" message.
  3. Get 1 "msg_type":"data:update" message from the server with all the data fields populated
  4. About 10 seconds later I get a
    ""msg_type":"data:error","tag":"xxxxx","value":"disconnected","error_type":"vehicle_disconnected"}"
  5. 60 seconds later my socket gets closed with CloseCode=1002 Reason="No more WebSocket frame from the server."

Really appreciate the help

@timdorr
Copy link
Owner Author

@timdorr timdorr commented Nov 12, 2020

On my MCU1 Model S, I get a single frame of data for my car's current position. I don't know the behavior of other vehicle and/or MCU versions. They may not emit any data if they are in deep sleep.

Unfortunately, I haven't yet gotten the MCU2 upgrade, so I can't verify any behavior differences. I would recommend trying it with the car in drive. If you can get someone else to drive it around for a bit while testing, that would be best.

@ktoonsez
Copy link

@ktoonsez ktoonsez commented Nov 12, 2020

Ok, I will do that, thanks again for all the help!

@mseminatore
Copy link

@mseminatore mseminatore commented Nov 12, 2020

@timdorr I had already made the change to use the bearer OAuth token from a successful login. That is why I am confused.

{
msg_type: "data:subscribe_oauth",
token: options.authToken,
value: options.values.join(','),
tag: options.vehicle_id.toString()
}

@mseminatore
Copy link

@mseminatore mseminatore commented Nov 12, 2020

@timdorr OK, belay that. in debugging through I discovered that further upstream I was not passing in the correct token. I am now receiving a data:update response. Thanks!

@ddaddy
Copy link

@ddaddy ddaddy commented Nov 12, 2020

@timdorr, thanks for posting the code up, finally figured out what I was doing wrong. I used the "id" from the vehicle list like you do on the /vehicle API instead of using the other "vehicle_id" and it now works. Not sure if it is as expected but I do get the following and maybe you could tell me if that is what happens?

**This is with the car parked and awake.

  1. Connect to the socket and get "{"msg_type":"control:hello","connection_timeout":0}"

  2. Send the "subscribe_oauth" message.

  3. Get 1 "msg_type":"data:update" message from the server with all the data fields populated

  4. About 10 seconds later I get a

""msg_type":"data:error","tag":"xxxxx","value":"disconnected","error_type":"vehicle_disconnected"}"

  1. 60 seconds later my socket gets closed with CloseCode=1002 Reason="No more WebSocket frame from the server."

Really appreciate the help

The socket only sends data when something changes and times out after 10 seconds.

So you get 1 hit of the current state, then updates after that.
Try turning your heating or air con on and you'll get more data as the power use changes. Or drive around so the speed and gps changes.

No data after 10 seconds, it times out. I can't remember if you have to reconnect or just send the subscribe again.

@ktoonsez
Copy link

@ktoonsez ktoonsez commented Nov 12, 2020

@ddaddy, It appears that if I keep pinging the server every 20 to 30 seconds with the connection it stays connected and then when I wake the vehicle out of a deep sleep I do get a "data:update". Im curious if there are any other values we can poll from the "value" field, charging would be nice so I could monitor when it starts and stops (mostly curious about stop since I set the start time).

@ddaddy
Copy link

@ddaddy ddaddy commented Nov 12, 2020

These are the ones I found:

speed,odometer,soc,elevation,est_heading,est_lat,est_lng,power,shift_state,range,est_range,heading,est_corrected_lat,est_corrected_lng,native_latitude,native_longitude,native_heading,native_type,native_location_supported

power will tell you if charging or not as it goes negative. Be wary that Regen braking also makes it go negative.

How are you pinging it? I tried all sorts of ping commands and none seemed to work.

@ktoonsez
Copy link

@ktoonsez ktoonsez commented Nov 12, 2020

Currently while testing I am using a library called "com.neovisionaries:nv-websocket-client" for android until I get this stuff setup. They have a func/property you can set called "setPingInterval", I set it to 20 seconds and it has kept me connected. So see if you can find a similar property in your socket.

@ktoonsez
Copy link

@ktoonsez ktoonsez commented Nov 12, 2020

@ddaddy, I added some function to monitor what the ping/pong was actually sending and it looks like it is just sending a rolling integer, starting with "1" (Ascii 49), and the server answers with the same integer.

******* EDIT*******
Turns out it was a fluke, I have tried it 3 more times and I never get any more data from the websocket, I do stay connected but never get any updates until I send a "data:subscribe_oauth" message. But at least I could stay connected if I wanted to, but not much point if you dont get any updates.

@ddaddy
Copy link

@ddaddy ddaddy commented Nov 13, 2020

Thanks, I'll try the ping.

Did you make sure something is changing for it to report? Open the Tesla app and switch the air con on or start charging. Or go to the car and open the door.

@ddaddy
Copy link

@ddaddy ddaddy commented Nov 13, 2020

Tried pings every 4, 5 & 8 seconds, I still get the vehicle_disconnected after 10 seconds and I have to resent the subscribe_oauth message

@ktoonsez
Copy link

@ktoonsez ktoonsez commented Nov 13, 2020

@ddaddy, Sorry I didnt explain that very well, I still get the vehicle_disconnected (if we didn't then that would mean the vehicle would be awake 24/7) but I can maintain the socket connection using the ping so that I can ask for an update with the data:subscribe anytime I want.

@ddaddy
Copy link

@ddaddy ddaddy commented Nov 13, 2020

Ah ok. How long have you kept the connection alive for?
I automatically send the subscribe message when I receive the vehicle disconnected message and it tends to stay alive for about 5 minutes.

@ktoonsez
Copy link

@ktoonsez ktoonsez commented Nov 13, 2020

I let it stay connected for 3 hours without issue.

@jdemeyer1
Copy link

@jdemeyer1 jdemeyer1 commented Dec 1, 2020

Greetings!

I have set up a client locally (Eclipse/Java) using the com.neovisionaries:nv-websocket-client library as did @ktoonsez . I am able to connect and get the control:hello message.

When I submit the data:subscribe_oauth JSON, the stream replies with "Can't validate token."

In the token field, I provide "Bearer < OAuthtoken value >". I use the same token for the vehicle_data API and receive a valid response from the API.
In the tag field, I provide the value found in the id_s field from the vehicle_data API.

I refreshed the token and created a new token; I receive the same error response from the stream.

I have an M3 purchased new and delivered in Mar 2020.

Thoughts or ideas on what I might try to get past this error?

Thanks!
Joe

3 Dec 2020 UPDATE
This has been solved.

I had been using "Bearer token_value" for the token. Do not use "Bearer " in front of the token. Second, I was using the value of id_s for the tag. I received data_update when I used the value in the vehicle_id field. Both of these values are a part of the /vehicle_data API endpoint.

Thanks!

@timdorr timdorr pinned this issue Jan 31, 2021
@ddaddy
Copy link

@ddaddy ddaddy commented Mar 12, 2021

Has anyone noticed any issues today with the streaming API?
After I receive the control:hello and I send my data:subscribe_oauth it disconnects. I've tried another 3rd party library and it does the same.

{"msg_type":"data:subscribe_oauth","tag":"<vehicle_id>","token":"<auth_token>","value":"speed,soc,power"}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet