Skip to content

Collaboration document for PSK Identity 02

Jump to bottom Edit New page
Colin Kuebler edited this page Aug 14, 2020 · 279 revisions

Here we will share and organize our findings and data from #483

Please help edit this document!

Much of this was bulk copy-pasted and needs to be transformed into a useful form

Findings (and comments that need to be edited into findings)

  • firmware OS SDK version did not change, but the hash and build time did
    • OS SDK ver: 2.0.0(e8c5810) compiled @ Jan 25 2019 14:26:04
    • OS SDK ver: 2.0.0(29f7e05) compiled @ Sep 30 2019 11:19:12
      • interesting that this build date corresponds with the release of tuya-convert 2.0, coincidence?
  • psk begins with 02 rather than 01
    • guessing this will tell us the psk algorithm version
  • disabled most of the serial debugging output
    • unfortunate as this was a useful reverse engineering resource
  • did not respond to our smartconfig procedure
    • it may use a new mechanism or additional restrictions have been imposed
  • MAC address in flash appears to be compared with actual device MAC
    • likely to foil reverse engineering attempts by running firmware on a dev board
  • new factory written key found in flash, "pskKey"
    • this may mean the encryption key will no longer be derived deterministically, big bummer
    • only found in @rsbob's backup, looks like @Farfar's was updated to this firmware and thus lacks this factory written key
      • this may mean there is still hope for devices that came with a lower firmware, since updating to this firmware would require fetching the key
      • this is a very interesting new endpoint: tuya.device.uuid.pskkey.get

It appears a unique preshared key is programmed into each device at the factory. Unlike before where the key could be derived from the TLS identity, now the identity is a hash of the device's id and MAC address. For those following along, the new identity derivation is:

'\x02' + 'BAohbmd6aG91IFR1' + sha256(prod_idx + mac_addr)

where prod_idx is the ASCII of the 8 digit product ID, and mac_addr is the MAC address of the device in hex, all lowercase, no punctuation. These parameters are all stored in the flash in a JSON blob at 0xfb000.

What changed was before the PSK was derived directly from the hint. Now it seems the key is preprogrammed at the factory, not computed on the device, and no information about it is ever sent to the server. The device sends its unique ID (basically MAC address) and the server figures out (presumably from a database) what the key should be. The device can request the server downgrade to the old authentication, but the server cannot ask the device to downgrade. Since we emulate the server, the downgrade path doesn't help us.

I tried to load the firmware backup onto a Node-MCU-like board, but I got this error

Yeah, they worked pretty hard on that. There a ton of checks and countermeasures to make running the firmware on anything other than the Tuya device very difficult. I'd assume the concern was counterfeit devices mooching off their cloud services. But as a result, getting this firmware running under a debugger is probably a real challenge. I relied on static analysis personally, so I didn't try it. They do many checks against the MAC address and hardware identifiers and each device is given a unique (?) API key, so if one was cloned, they could easily blacklist all of the cloned devices.

Curious constant (in tuya_tls.c, perhaps?)

Yes. It seems Tuya copied the xkcd getRandomNumber function (https://xkcd.com/221/). The "randomness" comes from a small fixed pool (the string you see there). There's a computation I don't fully understand at the beginning, but it effectively memcpys a substring of that pool to use as the random data for the TLS connection. Now I have no background in cryptography, but as far as I can tell, that is not actually exploitable for us because of the pre-shared key.

Happy to have so many smart people helping to reverse engineer this.

So that we do not spend time repeating efforts, make sure to post your findings here, and to read the entire thread. I have collected my original findings in this comment here. I can add important notes to this as we learn more.

Accordingly, it would be very helpful to refrain from "me too" comments to keep the thread focused on findings. Instead, consider using the +1 reaction. Please do not take offense if I hide your non-finding related comment.

@Fjuxx we have reversed the app in its entirety, which does provide useful information related to the provision process and cloud communication, however the part we are interested in, the PSK, is between the IoT device and the cloud only. Therefore efforts will be best spent reversing the firmware samples, as we can only glean so much information from the cloud.

the userland SDK has been out for quite some time. We are not interested in the pairing process as that has already been faithfully recreated. The missing piece is how to derive the PSK from the PSK identity, if that is even possible with the information we can obtain before a TLS connection is established.

the PSK is never sent to the client application. It is only between the device and the cloud. Again, the pairing process is not the issue. The device has already paired successfully by the time it makes registration calls.

I can add to that a bit. After analyzing some of the firmware dumps posted here, I've been able to extract the PSK from some of the devices and decrypt the packet captures. Looking at the decrypted TLS communication between the device and Tuya's servers, it would appear the PSK is not transmitted to the device during normal pairing on up-to-date devices. Even if it were, it would be encrypted with itself, so not very useful.

It would appear that outside of updating a vulnerable device to the new firmware, the PSK never leaves Tuya's servers. Since this communication is encrypted by the highest version of the encryption scheme that the device supports, this isn't too helpful. Currently we can only decrypt that communication if the device is on the old firmware or we have a firmware dumps from that particular device.

Note also, it would appear the PSK is unique to each device, not shared among an entire model (as far as I can tell). Perhaps most odd is that some of the affected devices in this thread have clearly undergone firmware updates, despite claims that they have never been used. Looking at the stored wifi credentials, there is no evidence of them having been online before. I'm guessing Tuya is doing something to offer updates to existing stock. This is a bit strange, considering you can fully flash the device over the exposed UART test points. The hidden prod_test mode (production test) may be related. I haven't fully reverse engineered how it works, but it seems to offer the ability to perform many configuration actions.

There are a few other approaches I've come up with, but so far none of them have worked out. I've still got one or two more to explore when I have some time.

If anyone has a really old firmware dump from before they implemented TLS/HTTPS, that might be useful.

EDIT: As an aside, there is currently unused code for what I think is certificate pinning in the Tuya SDK (the one on the device). Not sure why, but it's there.

I remember watching hak5 ages ago and they have a linux app that would just acept anyone trying to connect to there default ssid using any PSK it would just say yes and accept them could we use something like this work?

No. You are probably referring to the WiFi pineapple and Jasager, which are for spoofing WiFi networks. The PSK they refer to is the WiFi password. The PSK we care about is a different "password" used to talk to Tuya's servers. Unlike the situation Jasager targets, the server cannot tell Tuya devices to turn off encryption. The devices will reject any connection not secured with the TLS_PSK_WITH_AES_128_CBC_SHA256 cipher suite.

A. In this case we can extract the PSK from the firmware backup of that particular device, no? tuya-convert already does the backup, so if we could extract the device specific PSK, would that solve the issue?

Not exactly. In order for tuya-convert to perform the backup, it must first reflash the device with the intermediate firmware. This requires breaking the TLS connection with the server by obtaining the PSK. So while you can get the PSK from the backup, obtaining it over WiFi requires you have the PSK already.

B. By the "the device" you mean mobile app? How come unused code is part of the firmware and was not omitted by the compiler?

Nope. By "device" I do mean the smart device, i.e. light bulb, power switch, motion sensor, etc. Because the Tuya SDK compilation process uses statically linked libraries, it is actually possible for the compiler to eliminate unused functions. Doing to an exported symbol from a library requires using a compiler (technically linker) feature called "link-time optimization". I haven't verified this against my firmware dump yet, but I do not believe Tuya is using this feature.

C. why do they use PSK, why not do the certificate pinning to the cloud? if I were to develop a firmware, I would do so, what is stopping them?

The likely answer is that the public key cryptography necessary to validate a certificate or chain is too slow on the ESP8266. I'm not entirely certain how the Tuya device connection to the server handles current status (e.g. turning a light on), but if it is frequently opening a new TLS connection, this could result in unacceptable latency in controlling the device. By using a preshared key, they can avoid needing to validate the certificate and perform the Diffie–Hellman key derivation. Instead they skip directly to deriving the TLS master key via a single SHA256 hash operation.

To be absolutely clear, the challenge of this issue is mapping known information (ie gwId) to the PSK (pskKey), which as mentioned before is indeed unique to every device. As @iracigt points out, this is not possible to extract this from the firmware, because to obtain the firmware would require we already know the PSK (or open the device at which point OTA becomes moot).

The previous implementation leaked information through the PSK ID (the MD5 of the auzKey), but this latest implementation uses already public information (the SHA256 of the gwId) as the PSK ID.

It may be that the pskKey is totally random and only stored on Tuya servers and the smart device. If this turns out to be the case, there is no solution to our challenge and this strategy of flashing firmware OTA is no longer viable.

Please read the thread before posting questions ideas or information, and please move the side chatter to a side channel, as this thread is already getting unmanageably long with duplicate comments, uninformed questions and off topic chat. Unless we can keep the focus on solving the challenge (which again, is mapping known information to the PSK) I will have to lock this thread.

Some of the pskKey and auz_key extracted from the posted firmwares above:

https://github.com/ct-Open-Source/tuya-convert/issues/483#issuecomment-599560871 : {"mac_addr":"2462ab3989fe","prod_idx":"08488420","auz_key":"diI5mLzLQx5GBNCQQZsZ8J0dQLYMaAeT","pskKey":"Z9ir6z2hsTlRVJKwEZyqpfyIzfLyzkMBkwyGd","prod_test":false}

https://github.com/ct-Open-Source/tuya-convert/issues/483#issuecomment-570766710 : {"mac_addr":"c44f33bc1794","prod_idx":"65046664","auz_key":"tKzPU69mMe3ns8PmA5M2cAuUUDOtrTeA","pskKey":"yhULg57DUA3Uo1xTP5xhoI0C1kRpWQOwqjMO8","prod_test":false}

https://github.com/ct-Open-Source/tuya-convert/issues/483#issuecomment-594468592 : {"mac_addr":"98f4abc96ac3","prod_idx":"06402221","auz_key":"Yw0VAIvFe3aWlvdKEZfmBPg8xfQ3Jg4Q","pskKey":"kPJ6yZaAbTUqlk5fHrCN6DyWY2Flz9LLI49un","prod_test":false}

  • The alphabet for both auz_key and pskKey seem to be (a-z, A-Z, 0-9), which is not base64 (a-z, A-Z, 0-9, +, /). That would lead me to the guess, that they are not the result of any computation (like hashing the mac_addr or the like), because converting the binary result of such a computation into the above format is not straight forward.
  • Does anybody have an idea, why the length of the pskKey is 37? That sounds rather random.

We know that is it likely not base64, as we have plenty of samples and no instances of the +/ characters. It is possible that it is base62, but it seems more likely that this is a random string generated at the factory and stored only on the device and the cloud.

Procedures

The missing part is how to compute the PSK from the PSK identity, if that is possible with the information we have.

As @iracigt has said, the prod_idx is used to compute the PSK identity. This is handed to the server by the client during the handshake, so it is not secret information.

prod_idx is not secret information either, it is the first part of the gwId as you have noticed, @CHAZICLE. In fact the gwId is what is being hashed here, which is prod_idx + mac.

The point is that the previous implementation did leak secret information through the PSK identity, which we could use to compute the PSK. Now that it does not, our job is harder.

In order to solve this, we will need samples of actual communications between the client (smart device) and the server (Tuya cloud), along with a copy of that device's firmware. If anyone is able to provide both we may be able to crack this.

One possible procedure to accomplish this would be:

  1. Start with a new device
  2. Create a bridged hotspot using create_ap
  3. Start recording the network interface with tcpdump or WireShark
  4. Connect smart phone
  5. Use the vendor app to connect the smart device to the hotspot
  6. Disconnect and stop recording
  7. Open the device and read the firmware over serial using esptool.py
  1. Install create_ap 
git clone https://github.com/oblique/create_ap
cd create_ap
sudo make install
cd ..
  1. Setup a pass through AP (assuming your interface is wlan0)
sudo create_ap wlan0 wlan0 MyAccessPoint MyPassPhrase
  1. Start recording
tcpdump -i wlan0 -w capture.pcap
  1. Connect your phone to MyAccessPoint (or whatever you decide to call it)
  2. Use the app (SmartLife or vendor branded app) to pair the device
  3. Wait for registration to complete
  4. Disconnect the device
  5. Go back to tcpdump and press Ctrl + C
  6. Disassemble the device and connect to the serial port of the ESP
  7. Download the firmware using esptool 
esptool.py read_flash 0 0x100000 firmware.bin
  1. Upload both capture.pcap and firmware.bin

Get: https://github.com/espressif/esptool Issue command: esptool.py read_flash 0 0x100000 firmware-backup.bin

Could we get some insight into whether the new PSKs are being randomly generated by flashing a device back and forth between old and new patched firmwares?

For example:

  1. take a device with pre-patched Tuya firmware and back it up
  2. register the device to the app and get the updated patched firmware
  3. backup the new firmware to determine the new PSK
  4. flash the device back to the pre-patched firmware
  5. register the device again to the Tuya app to get the patched firmware
  6. backup the new firmware, extract the PSK and determine if it matches the PSK from step 3)

This assumes you have a device you can physically flash and that it's possible for us to flash a device with an older version of the Tuya firmware.

It may be when a device is first updated through the app, that the new randomly generated PSK is linked to the device MAC (or other device identifier), and that same PSK is issued again after downgrading and updating. Or a random PSK is generated each time.

To clarify: As I understood, we need a device with a firmware that can be converted but isnt yet or one that has been converted + a backup of its original backup availiable. Then follow the steps of @finnzz . Devices that can not be patched are around and afaik dont help.

Yeah, a number of people have talked about ordering a group of plugs or bulbs, etc. And said that only some flashed with TuyaConvert (old Tuya firmware) while the others gave the PSK error (new Tuya firmware)

In this case you know there is a patched firmware available, which may also be waiting in the Tuya app for devices still on the old firmware. And if you can open the device up to serially flash, then you can run through the procedure.

As @Schlabberanesi was saying, another option is that you have a device with Tasmota/ESPHome, you saved the stock firmware, and you know the device has the patched firmware waiting for it in the Tuya app. You could then serially flash that Tasmota/ESPHome device back to stock and go through the process of ungrading/downgrading described above.

Edit: The question here is how are devices with old firmware being integrated into Tuya's newer security scheme. And can this help us understand how these new PSKs are created.

@ratsputin Only if you are able to open the bulb up to serial flash it. You would need to serial flash it to put it back to stock anyway. And once you register the bulb in the Tuya app and update the firmware, the only way to revert to the old firmware will be with a serial flash.

So if you can, then go through this procedure: https://github.com/ct-Open-Source/tuya-convert/issues/483#issuecomment-663500470 And maybe we will get lucky and learn something from comparing the backups.

Just to summarize what would be needed:

  1. need a Tuya device with old firmware, that works with TuyaConvert
  2. a patched firmware has to exist for that device model, and be available in the Tuya app for you to get.
  3. you need to be able to open the device up to serial flash it.

Data

Known Affected Devices

  • BSD34 smart socket
  • BSD29
  • Deltaco SH-P01E
  • Bakibo TB95
  • Bakibo TP22Y
  • MoesHouse Smart Downlight
  • loratap sc500w curtain switch
  • Teckin SB50
  • Gosund 800l bulb
  • Merkury MI-EW003-999W LED Light strip
  • DETA Quad Smart Switch (6904HA)
  • DETA Triple Smart Switch
  • Arlec GLD060HA lamp
  • AOFO Smart Power Strip (ZLD-44EU-W)
  • Gosund SP 112
  • Arlec PC399HA
  • Etersky WF-CS01 Curtain Switch
  • Gosund WB4 bulb
  • NX-SM400
  • Jinvoo SM-PW713
  • LOHAS E14 bulb
  • BW-LT29
  • NX-SM112
  • Zemismart 6 inch 14W WiFi RGBCW

Firmware

5 firmware backups that tuya convert made. One from a smart plug and another four from smart switches.

firmware-9c52d8.bin.gz firmware-52a6fb.bin.gz firmware-a068f4.bin.gz firmware-a0709d.bin.gz

Network Captures

Strings

OS SDK ver: 2.0.0(29f7e05) compiled @ Sep 30 2019 11:19:12
[N]%s:%d metedata is without encryption, cover this partition with encrypted data
[N]%s:%d var block [%d] last version is without encryption,now need to encrypt data and cover this partition
[ERR]%s:%d flash_aes128_ecb_encrypt err
[ERR]%s:%d !!!!!!CHIP_MAC:%s FLASH_MAC:%s!!!!!!
aes-internal-dec.c
aes-internal-enc.c
lhttps://a3.tuyacn.com/gw.json
https://a3.tuyaeu.com/gw.json
https://a3-ueaz.tuyaus.com/gw.json
https://a3.tuyaus.com/gw.json
{"mac_addr":"500291e8f141","prod_idx":"26636676","auz_key":"wg7xdxUcsi7W0EmnPUlEtuwoyZwcF2W2","pskKey":"GUKgTWzsxGCn8UR5tIFFluuKD2qA8FYmaZwY2","prod_test":false}
{"ap_ssid":"SmartLife","ap_pwd":null}
{"CC":"CN"}
vtrust-flash
ESP_E8F141
vtrust-flash

{"ip":"10.42.42.16","gwId":"22885450c44f33b5ed3e","active":2,"ability":0,"mode":0,"encrypt":true,"productKey":"key75ugrk53wuycc","version":"3.3"}
{"mac_addr":"c82b964e3b29","prod_idx":"64785071","auz_key":"eDOnWtAUy5gJOiKtcXkNplBMqOMjJmM3","pskKey":"ggD99i71cC7R7MTVnHadCtjaK28H2f6fOJLrG","prod_test":false}{"mac_addr":"c82b964e3b22","prod_idx":"64785071","auz_key":"in18hm8DJch2NzlkjkSVHVuFJ3kpUo6k","pskKey":"hcXNpRfvnob0zqZWQ4zL4oxa0BHHRzC6jns3X","prod_test":false}

TLS-PSK-WITH-AES-128-CBC-SHA256 {"mac_addr":"2462a______a","prod_idx":"648______2","auz_key":"hKz8MCoerc______5vD","pskKey":"yrbbNk______rx3FfRQnRfXHs3rC8","prod_test":false}

Add a custom footer
Add a custom sidebar

Clone this wiki locally