New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No token in miio2.db after version 5.0.29 of MiHome app... #185

Open
khalitovsv opened this Issue Jan 27, 2018 · 27 comments

Comments

Projects
None yet
@khalitovsv
Copy link

khalitovsv commented Jan 27, 2018

In versions 5.0.31+ there is empty field token in miio2.db file.
They change something in application.
An investigation is required.

Tested on Mi Robot Vacuum Cleaner (1st generation)

@rytilahti

This comment has been minimized.

Copy link
Owner

rytilahti commented Jan 27, 2018

Could you check if there's another field which could have the token? Maybe they have done the same or similar as with the iphone app and encrypted it? Do you see is_encrypted or similar boolean field?

@khalitovsv

This comment has been minimized.

Copy link

khalitovsv commented Jan 27, 2018

There is content of miio2.db in JSON format (PRIVATE is private information, changed by me):

[
{
"bssid": "00:24:01:CF:1B:C1",
"canAuth": "1",
"canUseNotBind": "0",
"desc": "Charging:89%",
"descNew": "",
"descTimeJString": "",
"did": "55393227",
"eventInfo": "{"event.back_to_dock":"{\"timestamp\":1517070715,\"value\":[0]}","event.back_to_dock_no_power":"{\"timestamp\":1493405918,\"value\":[0]}","event.back_to_origin_fail":"{\"timestamp\":1495822059,\"value\":[0]}","event.back_to_origin_succ":"{\"timestamp\":1517000558,\"value\":[0]}","event.clean_complete":"{\"timestamp\":1517070666,\"value\":[0]}","event.error_code":"{\"timestamp\":1517070715,\"value\":[0]}","event.relocate_fail":"{\"timestamp\":1493819748,\"value\":[0]}","event.robot_timed_ended":"{\"timestamp\":1506240979,\"value\":[1506240960,0]}","event.robot_timed_started":"{\"timestamp\":1512226801,\"value\":[1512226800,0]}","event.status":"{\"timestamp\":1517071680,\"value\":[{\"battery\":89,\"clean_area\":8410000,\"clean_time\":1207,\"dnd_enabled\":0,\"error_code\":0,\"fan_power\":90,\"in_cleaning\":0,\"map_present\":1,\"msg_seq\":913,\"msg_ver\":6,\"state\":8}]}","prop.back_to_origin_fail":"0","prop.back_to_origin_succ":"0","prop.clean_complete":"0","prop.error_code":"0","prop.fan_power":"90","prop.ota_progress":"100","prop.ota_progress_ts":"1517002835","prop.ota_state":"idle","prop.ota_state_ts":"1517014037"}",
"extraInfo": "{"isSetPincode":0,"fw_version":"3.3.9_003094","needVerifyCode":0,"isPasswordEncrypt":0,"is_factory":false}",
"index": "63",
"isOnline": "1",
"latitude": "PRIVATE",
"localIP": "192.168.1.46",
"location": "1",
"longitude": "PRIVATE",
"mac": "28:6C:07:F7:3C:5C",
"methodInfo": "",
"model": "rockrobo.vacuum.v1",
"name": "R2D2",
"ownerId": "",
"ownerName": "",
"parentId": "",
"parentModel": "",
"permitLevel": "16",
"pid": "0",
"propInfo": "",
"resetFlag": "0",
"rssi": "-66",
"showMode": "1",
"ssid": "PRIVATE",
"token": "",
"userId": "PRIVATE",
"version": ""
}
]

Finding of actual token in HEX or BIN representation in /data/data/com.xiaomi.smarthome, gave the only one result: some cache files contains HEX token in /data/data/com.xiaomi.smarthome/cache/smrc4-cache folder

When downgrading to version 5.0.29, we can find actual token in miio2.db file again...

@yawor

This comment has been minimized.

Copy link
Contributor

yawor commented Jan 27, 2018

I've installed 5.1.0 and I can confirm that. I've analysed the decompiled code and it seems that the token is no longer being kept locally, but instead it's being downloaded from the server on app opening. This can be observed when opening app after forcefully closing it. Previously, the device would show up immediately in the UI. Now it takes a moment, probably the app waits for the server response containing a list of devices, which contain tokens amongst other information.

@yawor

This comment has been minimized.

Copy link
Contributor

yawor commented Jan 28, 2018

For few past days I've been playing around the communication between MiHome app and xiaomi servers. I've captured the traffic using the mitmproxy. The whole communication is using SSL so there's some setup required for the capture to work.
In addition to SSL, after the authentication most of the data in requests and responses is additionally encrypted and then encoded in base64. I already know what encryption algorithm is in use and I've been able to decrypt most of the communication (there are still some bits and pieces I'm not sure about). I can confirm that the token is being received from the server (it was even in earlier versions).
It seems that tapping into the app's communication may be the only way to retrieve the tokens with versions 5.0.31+. The issue is that setting up the mitmproxy may to too much for most miio users. I'm thinking about simulating the app in the library: a cli command that would ask a user about the email and password for mi account used in the app and the library would use them to authenticate to xiaomi servers and request the devices list associated with the account.

@rytilahti

This comment has been minimized.

Copy link
Owner

rytilahti commented Jan 28, 2018

Interesting findings, and it's very unfortunate that it's being made more difficult to obtain those tokens. Maybe we should add a note into the README file to warn about that. I'm wondering whether this change is done in the device specific communication part or in the main app. Can one downgrade the main app and regain the access, or will it be necessary to have an old version of the vacuum plugin available?

To the cli tool with emails & password. The problem with such a cli tool would be that it needs to be updated regularly (assuming they want to block that way) depending on how they are blocking it. I'm not completely against it though.

For cloud<->vacuum communication you may want to check https://github.com/dgiese/dustcloud - they have pretty much reverse-engineered it afaik.

@yawor

This comment has been minimized.

Copy link
Contributor

yawor commented Jan 28, 2018

The change is at the main app level. Downgrades are possible now, but I wonder if they can somehow block it. For example they could make newer versions of plugins compatible only with newer app versions and disable access to older plugin versions for new installations.

Regarding accessing the Xiaomi servers from the library, if the request could be made indistinguishable from those made by the app, then to block it Xiaomi would need to change something in the protocol, which in turn would require a new app version and immediate cut off of all their clients until they upgrade. But I partially share your concern, that's why I haven't yet disclosed the details of the protocol.

Thanks for the link. I've been looking for some information regarding the communication between the device and the servers. I've seen that the protocol is the same but I don't know the token. But there's really nothing interesting in the payload anyway, when I look at the info on the linked page.

BTW I see that Air Purifier Pro responds to miIO.ota, miIO.get_ota_state and miIO.get_ota_progress commands. It would be nice to capture the moment Xiaomi pushes the update and get the original firmware file like they did for the vacuums.

@rytilahti

This comment has been minimized.

Copy link
Owner

rytilahti commented Jan 28, 2018

Ok, I think it'd be fine to provide that functionality for fetching the tokens online (not just for the vacuum, but also for the other linked devices). It'd need finding out what type of requests the app is making (e.g. is there User-Agent and does it depend on version of the app or the OS behind etc.).

I think they analyzed how the token for server<->device communication is delivered, but didn't pay much attention to it. Support for those commands is being worked in #153 - the same update procedure is apparently used for at least most of the other devices too, so that's why I started to implement it as a part of the Device. Those commands are only accepted by the device when it has not been provisioned or when you have access to the device<->cloud token.

@rytilahti

This comment has been minimized.

Copy link
Owner

rytilahti commented Feb 2, 2018

BTW I see that Air Purifier Pro responds to miIO.ota, miIO.get_ota_state and miIO.get_ota_progress commands.

Could you post and/or implement a PR for the responses from those? Looks like the vacuum responds only to miIO.ota (tested by updating the FW), but nothing on those two others. Maybe I'm missing some required parameter (didn't namely pass any)?

@syssi

This comment has been minimized.

Copy link
Collaborator

syssi commented Feb 2, 2018

@rytilahti Do you use the mihome app and could you provide the plugin (apk) of the vacuum cleaner?

@rytilahti

This comment has been minimized.

Copy link
Owner

rytilahti commented Feb 2, 2018

@syssi sure, let's discuss on discord.

Looks like those commands work just fine now. get_ota_progress reports [percentage], and get_ota_state has values idle, downloading, installing and failed, at least.

@crhan

This comment has been minimized.

Copy link

crhan commented Feb 16, 2018

have the same issue. Does anyone have any solution?

I try to reset wifi configure and obtain the token. But it seems like after configure the wifi. The token will reset again.

@peletiah

This comment has been minimized.

Copy link

peletiah commented Apr 8, 2018

I found it easiest to get the token directly from the robot, with the modified firmware from the dustcloud-project you can simply login via ssh and do
printf $(cat /mnt/data/miio/device.token) | xxd -p

Here are the step-by-step instructions to get a rooted firmware: https://github.com/dgiese/dustcloud/wiki

@WeslyG

This comment has been minimized.

Copy link

WeslyG commented May 27, 2018

I was able to get the key to my lamp yeelink RGB

  • download mi home (5.0.29)
  • backup app
  • open miio2.db and find token
  • install last version from google play

now I'm afraid of letting the children into my house, that they would not press 5 times on my switch, and do not restet my lamp)

@yawor

This comment has been minimized.

Copy link
Contributor

yawor commented Aug 18, 2018

I got some private questions regarding the way Xiaomi app encrypts the communication with the Xiaomi servers. This is actually only one of the schemes. I've seen some other too, but didn't bother with them yet.

The app sends POST requests to https://sg.api.io.mi.com/app/*. The Content-Type should be application/x-www-form-urlencoded and the request body should be URLEncoded form, which contains at least following form fields: data, rc4_hash__, signature, _nonce and ssecurity.
The rc4_hash__ gives the hint that the app uses RC4 symmetric encryption. In reality an RC4 Drop is used. This is a modification of RC4, where the encoder drops the result of encrypting/decrypting (it's symmetric so encryption and decryption are the same operation) N bytes which are fed to the algorithm before the real data is passed.

An RC4 key needs to be calculated first. The key is calculated using values of _nonce and ssecurity values. Both are stored as base64 and need to be decoded first to raw values. The key is a SHA256 hash (raw) of value created by concatenating raw values (decoded from base64) of ssecurity and _nonce. The key is different for each request.
To decrypt either the data in request or body of the response, first it needs to be decoded from base64 to raw data. The RC4 decoder should be initialised with the key calculated from the request and then it should first be used to encode 1024 bytes, all equal to 0 (the result of that should be dropped, hence the RC4 Drop) and then it can be used to properly decode the request or response data. It must be initialised that way separately for request and response.

I hope the description is understandable. I've only focused on decrypting the traffic. I haven't worked on encrypting my own data and simulating the app. For that some other data is needed: for example the cookie, which is send as a request header - it needs to be obtained during a proper authentication, which is something I haven't been able to fully understand yet, as the app seems to be authenticating and authorising with multiple services.

@hrbonz

This comment has been minimized.

Copy link

hrbonz commented Aug 27, 2018

Has any progress been made here? I was under the impression that the lib was still working and bought one of those plugv3.
I wouldn't mind working on a PR if I'm pointed in the right direction, that last comment gives some clues as to where I should be looking.

I'd suggest to add a warning at the top of the README to mention that issue.

@syssi

This comment has been minimized.

Copy link
Collaborator

syssi commented Aug 27, 2018

Please downgrade the android app and retrieve the token.

@hrbonz

This comment has been minimized.

Copy link

hrbonz commented Aug 28, 2018

It's possible that way but doesn't sound like a good solution in the long run, I don't think iOS users can do that simply for example.

Is there something that makes contacting the Xiaomi back end impossible or very difficult? I'll do some research here, it doesn't seem impossible but I might be missing something.

As a first step, I'll push a PR with a warning in the README. Right now the documentation is lacking on the subject.

@rytilahti

This comment has been minimized.

Copy link
Owner

rytilahti commented Aug 28, 2018

Well, 3rd party apps such as the one from flole does contact the cloud, so I don't think they are trying to actively make it hard to do. A PR for the current situation sounds good, a separate "webdiscover" discovery module for fetching all the tokens from registered devices would be really useful in long term (which was also proposed by @yawor above).

@yawor

This comment has been minimized.

Copy link
Contributor

yawor commented Aug 31, 2018

@rytilahti I though that maybe I'll be able to retrieve some info from the flole app, but their app doesn't connect to xiaomi servers directly. Instead it connects to flole's server, which probably does all the communication with xiaomi servers.

@marcelrv

This comment has been minimized.

Copy link

marcelrv commented Aug 31, 2018

@yawor

This comment has been minimized.

Copy link
Contributor

yawor commented Aug 31, 2018

Unfortunately I won't have time anytime soon to look further into this. If anyone wants to try analysing traffic between the app and Xiaomi cloud, then the best approach is to use a mitmproxy software. It's written in Python and it can be used on different systems.
The app uses SSL, so some steps need to be taken before it's possible to look at the traffic. The mitmproxy generates its own CA certificate. It needs to be installed on the phone for the duration of the session. It should be uninstalled after, because it's not safe to leave non-trusted CA certificates on the phone. Then start the mitmproxy and change proxy settings for WIFI connection in the phone to use the computer with mitmproxy running as a proxy (the computer needs to be on the same network of course). For 100% certainty that all the traffic is captured, the mobile connection should be turned off.
If everything is set up correctly, mitmproxy should start displaying requests and responses, https included. The capture session can be saved to a file for later analysis.
It's best to clear the MiHome app data before the capture and start the cleared app after setting up the mitmproxy. That way it's possible to capture everything the app does.

@royduin

This comment has been minimized.

Copy link

royduin commented Oct 23, 2018

Any updates on this? As mentioned in the docs of Home Assistant: https://www.home-assistant.io/components/vacuum.xiaomi_miio/#retrieving-the-access-token I've first tried to extract the token from the latest version which was installed on my phone. No token, so I've removed the app and downgraded to 5.0.30 but the app is continuously crashing on my OnePlus 3T. After multiple tries I finally managed to login but my devices where gone. Tried to re-add them but it's impossible to add a new devices. No devices are listed.

@royduin

This comment has been minimized.

Copy link

royduin commented Oct 23, 2018

Nevermind, came across this post: https://community.home-assistant.io/t/for-those-who-cant-find-the-xiaomi-vacuum-token/74631 so I downloaded 5.0.19 instead of 5.0.30, logged in and after a restart my Vacuum was visible. Repeated the steps and got my token.

@fluefiske

This comment has been minimized.

Copy link

fluefiske commented Oct 24, 2018

@syssi

This comment has been minimized.

Copy link
Collaborator

syssi commented Oct 24, 2018

Uninstall the old version of the app and install the one new one afterwards (play store).

@wernerhp

This comment has been minimized.

Copy link

wernerhp commented Nov 5, 2018

Here's what I did for philips.light.bulb to obtain a token.

  1. Reset the bulb by turning it on and of 5 times.
    The bulb will flash 5 times.
  2. Connect to the bulb's Wifi using your computer, then run miio discover
    Take note of the Token
Device ID: xxxxxxxx
Model info: philips.light.bulb
Address: 192.168.4.1
Token: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx via auto-token
Support: At least basic
  1. Add the bulb to the Mi Home app to register it with your WiFi.
@fluefiske

This comment has been minimized.

Copy link

fluefiske commented Nov 5, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment