Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for devices using the new TLS SDK #279

Merged
merged 40 commits into from Oct 2, 2019

Conversation

kueblc
Copy link
Collaborator

@kueblc kueblc commented Sep 27, 2019

Huge thanks to @M4dmartig4n for his work reverse engineering and developing a PoC (#273). Would not be possible without him.

This PR incorporates @M4dmartig4n's PSK proxy which enables newer Tuya devices to communicate with tuya-covert over HTTPS/MQTTS, along with other changes to maintain compatibility with older firmware and improve compatibility with newer firmware.

  • Add PSK proxy
  • Add encrypted responses
  • Calculate HMAC signature for firmware files
  • Extend schema to prevent some devices crashing
  • Add command line option to set secKey
  • Add protocol 2.2 support via command line flag to mq_pub_15
  • Updated README and prerequisites installer

Please test and provide feedback here, thanks!

M4dmartig4n and others added 30 commits September 18, 2019 21:38
Added optional argument "--protocol" which specifies whether to use the old or new MQTT encoding
Cleaned up a bit, adding some descriptive variable names and reducing code clutter
Attempted to integrate new API responses with old ones, based on et=1 param
**Untested**
Print request details, method, headers, and decrypted payload
Print response payload
Removed additional config.get payload, does not appear to be necessary and in some cases was causing problems
Revert upgrade type to 0 since we are trying to update the firmware on the main ESP82xx, not an external MCU
Tidying up by removing some redundant endpoints and adding comments
Now all but the activation endpoints use encryption when requested
…if schema is shorter than expected

Added missing URLs in activation response
Readded .updatestatus to avoid .upgrade from catching this endpoint
Verify payload decryption is JSON
@amiranees
Copy link

Also this Kali setup, I can ssh to it like the raspbian ?

@NicolaiVdS
Copy link

@amiraness and yeah user pass are root:toor

@kasav81
Copy link

kasav81 commented Oct 3, 2019

Tried new-api and official tuya-convert after new-api was merged with Gosund SP112 (identical to Blitzwolf SHP5) but did not succeed, logs attached:

smarthack-mqtt.log
smarthack-psk.log
smarthack-smartconfig.log
smarthack-web.log
smarthack-wifi.log

After that I opened the Plugs and flashed them using an usb serial converter, I don't know if this is of any use to you but here is a backup of the original FW:

fwbackup.zip

@kueblc
Copy link
Collaborator Author

kueblc commented Oct 4, 2019

Thanks @bruvv and @kasav81 for the feedback, this should be fixed with #289.

@dasb00ter
Copy link

dasb00ter commented Oct 4, 2019

Allright I have made some progress. I had tried to upgrade my tuya-convert 1 to tuya-convert 2 and I supposed that caused issues even though the install script worked flawlessly.
This time I started from scratch with a new raspberry pi buster lite image and rebuilt it according to digiblurs tut.

I have flashed 3 of 4 teckin sp20's but one seems to be bricked pre flashing tasmota. It seems to have got stuck somewhere in the coming online process and I cant get it to continue. When I plug the device in sometimes I get a blue purple light for a moment but sometimes I get a slow blue flashing light. The button does nothing and it doesn't matter when I press it or how long I press it, it wont go into pairing mode. Not sure where it is stuck or how I can proceed. Can somebody help me out. I have tried to get it going in the app too but no dice

(ps by the way my succesfull flashing sequence had the led solid blue then red and wait on that for awhile like a full page of dots with putty expanded to screen size and then the back up begins)

@WolfgangHusen
Copy link

Hi there,

I tried with the Teckin SP21. It seemed to me the flashing part worked. But now the Plug is not responding or showing any (blue) lights at all...

Here the output (sorry for the strange formatting):

`======================================================
TUYA-CONVERT

https://github.com/ct-Open-Source/tuya-convert
TUYA-CONVERT was developed by Michael Steigerwald from the IT security company VTRUST (https://www.vtrust.de/) in collaboration with the techjournalists Merlin Schumacher, Pina Merkert, Andrijan Moecker and Jan Mahn at c't Magazine. (https://www.ct.de/)

======================================================
PLEASE READ THIS CAREFULLY!

TUYA-CONVERT creates a fake update server environment for ESP8266/85 based tuya devices. It enables you to backup your devices firmware and upload an alternative one (e.g. ESPEasy, Tasmota, Espurna) without the need to open the device and solder a serial connection (OTA, Over-the-air).
Please make sure that you understand the consequences of flashing an alternative firmware, since you might lose functionality!

Flashing an alternative firmware can cause unexpected device behavior and/or render the device unusable. Be aware that you do use this software at YOUR OWN RISK! Please acknowledge that VTRUST and c't Magazine (or Heise Medien GmbH & Co. KG) CAN NOT be held accountable for ANY DAMAGE or LOSS OF FUNCTIONALITY by typing yes + Enter

yes

Starting AP in a screen
Stopping any apache web server
Starting web server in a screen
Starting Mosquitto in a screen
Starting PSK frontend in a screen

======================================================

IMPORTANT

  1. Connect any other device (a smartphone or something) to the WIFI vtrust-flash
    The wpa-password is flashmeifyoucan
    This step is IMPORTANT otherwise the smartconfig will not work!
  2. Put your IoT device in autoconfig/smartconfig/pairing mode (LED will blink fast). This is usually done by pressing and holding the primary button of the device
  3. Press ENTER to continue

======================================================
Starting pairing procedure in screen
RTNETLINK answers: File exists
~/tuya-convert
Waiting for the upgraded device to appear
If this does not work have a look at the '*.log'-files in the 'scripts' subfolder!
.....................................
IoT-device is online with ip 10.42.42.42
Fetching firmware backup
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1024k 100 1024k 0 0 54355 0 0:00:19 0:00:19 --:--:-- 31870
curl: Saved to filename 'firmware-33ba82.bin'

Getting Info from IoT-device
VTRUST-FLASH 1.1
(c) VTRUST GMBH https://www.vtrust.de/35c3/
READ FLASH: http://10.42.42.42/backup
ChipID: 33ba82
MAC: CC:50:E3:33:BA:82
BootVersion: 7
BootMode: normal
FlashMode: 1M DOUT @ 40MHz
FlashChipId: 144068
FlashChipRealSize: 1024K
Active Userspace: user2 0x81000

======================================================
Please make sure to note the correct SPI flash mode!
Installing an alternative firmware with the wrong flash mode will leave the ESP unable to boot!

Next steps:

  1. To go back to the orginal software

    curl http://10.42.42.42/undo

  2. Be sure the conversion software runs in user2

    curl http://10.42.42.42/flash2

  3. Flash a third party firmware to the device
    BE SURE THE FIRMWARE FITS THE DEVICE AND USES THE CORRECT FLASH MODE!
    MAXIMUM SIZE IS 512KB
    put or link it to ./files/thirdparty.bin
    A basic build of Sonoff-Tasmota v6.5.0 is already included in this repository.

    curl http://10.42.42.42/flash3

Alternatively let the device download and flash a file via HTTP:

curl http://10.42.42.42/flash3?url=http://10.42.42.1/files/thirdparty.bin

HAVE FUN!
wolfgang@Abacab:/tuya-convert$ curl http://10.42.42.42
VTRUST-FLASH 1.1
(c) VTRUST GMBH https://www.vtrust.de/35c3/
READ FLASH: http://10.42.42.42/backup
ChipID: 33ba82
MAC: CC:50:E3:33:BA:82
BootVersion: 7
BootMode: normal
FlashMode: 1M DOUT @ 40MHz
FlashChipId: 144068
FlashChipRealSize: 1024K
Active Userspace: user2 0x81000
wolfgang@Abacab:
/tuya-convert$ curl http://10.42.42.42 > teckin1.txt
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 273 100 273 0 0 383 0 --:--:-- --:--:-- --:--:-- 383
wolfgang@Abacab:/tuya-convert$ curl http://10.42.42.42/flash2
Device is already booting from userspace 2 (0x81000)
wolfgang@Abacab:
/tuya-convert$ curl http://10.42.42.42/flash3
Device should flash http://10.42.42.1/files/thirdparty.bin and restart
`

@floli
Copy link

floli commented Oct 5, 2019

@WolfgangHusen Please use triple backticks (```blockquote```) to

use block-preformatted text

Furthermore, do you see a WiFi network vtrust-recovery? If yes, you might run into a similar issue as I, #285

@pfeerick
Copy link

pfeerick commented Oct 5, 2019

@WolfgangHusen Triple ``` on lines of their own before and after the log messages should fix that formatting ;) Single ` is for short inline snippets. I'm curious though... when I did all my tuya devices, I skipped flash2, and went straight to flash3... was I just fortunate?

@bruvv
Copy link

bruvv commented Oct 5, 2019

@NicolaiVdS @kueblc installed kali on my pi 3b but that doesnt start the AP vtrust-flash it hangs on: Applying iptables rulesz when isuing the command sudo iptables --flush it just hangs on kali.... acording to:
https://forums.kali.org/showthread.php?43649-Solving-iptables-issue-on-Kali-Linux-2019-1-ARM-32-bit-image-(kalitorify-kali-anonsurf)
Using the command iptables-legacy does work in kali
using the command:

root@ws-54879:~# mv /usr/sbin/iptables /root/scripts/
root@ws-54879:~# ln -s /usr/sbin/iptables-legacy /usr/sbin/iptables

Fixed the iptables for me.

@bruvv
Copy link

bruvv commented Oct 5, 2019

Alright KALI didn't do it for me. But I finally got all my TUYA devices flashed! What I did was install rasbian light and start with sudo apt-get install network-manager and than run the rest.

@kueblc
Copy link
Collaborator Author

kueblc commented Oct 5, 2019

I tried with the Teckin SP21. It seemed to me the flashing part worked. But now the Plug is not responding or showing any (blue) lights at all...

@WolfgangHusen if the flashing was successful, which it looks like it was, the device will not show any lights or respond to anything because it's not configured yet. Look for a sonoff-**** SSID that you can connect to and configure through 192.168.4.1.

I'm curious though... when I did all my tuya devices, I skipped flash2, and went straight to flash3... was I just fortunate?

@pfeerick you'll only need to run /flash2 if you are not already running in userspace2. It wouldn't let you do /flash3 otherwise, so there is nothing to worry about.

@bruvv did you try #225?

@bruvv
Copy link

bruvv commented Oct 5, 2019

@kueblc no but managed to flash smart plugs, led strip just 1 light to go which is probably not ESP compatible.

@amiranees
Copy link

@bruvv did you managed to flash the action bulbs?

@bruvv
Copy link

bruvv commented Oct 5, 2019

@amiranees no the E27 isn't esp controlled according to the logs.

@amiranees
Copy link

@bruvv so al this time we assume we could but it wasn't possible? Damn :)

@bruvv
Copy link

bruvv commented Oct 5, 2019

Didn't take it apart yet ;) so cannot say for sure. But I did manage to flash the led led strip and the smart wall plugs tho

@WolfgangHusen
Copy link

I tried with the Teckin SP21. It seemed to me the flashing part worked. But now the Plug is not responding or showing any (blue) lights at all...

@WolfgangHusen if the flashing was successful, which it looks like it was, the device will not show any lights or respond to anything because it's not configured yet. Look for a sonoff-**** SSID that you can connect to and configure through 192.168.4.1.

thanks @kueblc: I must have been blind. I can confirm Teckin SP21 works fine.

@WolfgangHusen
Copy link

today I flashed a Gosund SP111. Went also fine (even though I had to restart the flashing process several times before the plug was identified).
So thanks for the great work!

@dasb00ter
Copy link

dasb00ter commented Oct 8, 2019

Ok as mentioned I have one sp20 plug that went through the flashing process but seems to be bricked. It produces no ssid that I can see. When I plug it in it flashes blue slowly for awhile then stops. It cannot be reached or paired in the tuya smart life.

Can anybody help

@joshgetter
Copy link

I was able to flash 4 Feit electric rgb bulbs from Costco the other day with this update.

@sam-ward
Copy link

I have been trying to get tuya-convert running on my rPi 3b+ (with raspbian buster) to flash my Brilliant smart plugs. I originally hit a brick wall with an earlier version of the code as they seem to come with a firmware that expects https.

Since I saw this version out I thought i'd try again. Everything seems to spin up correctly, and I can connect my phone to the vtrust-flash AP. However when I put the plug into pairing mode, it doesn't seem to want to connect. On further investigation I found the following lines in the smarthack-psk.log (repeated adnauseaum):

new client on port 443 from 10.42.42.16:1549 ID: b'0142416f68626d64366147393149465231d52eac53ae672aebb9436e23c3cef9cd5fc6f7cdbc3bdef91aad447e3ad6ebe905' could not establish sslpsk socket: <class 'ssl.SSLError'> returned a result with an error set

Any suggestions on where further I should look or what else I could try?

Cheers.

@kueblc
Copy link
Collaborator Author

kueblc commented Oct 14, 2019

@sam-ward did you run install_prereq again once updating?

@sam-ward
Copy link

@sam-ward did you run install_prereq again once updating?

Because I had been messing around with the image so much, I started with a freshly formatted SD card. All I had done was change the hostname, apt-get dist-upgrade, and apt-get install git.

@kueblc
Copy link
Collaborator Author

kueblc commented Oct 14, 2019

Is python --version 2.7x? Do you have python2? Try changing the first line of psk-frontend from python to python2

@sam-ward
Copy link

Is python --version 2.7x? Do you have python2? Try changing the first line of psk-frontend from python to python2

That did the trick thank you. The system default must have been changed somewhere along the line. After doing that I also had to rerun install_prereq, as the python2 was missing the sslpsk module. But I have it flashed successfully now.

@depen84
Copy link

depen84 commented Oct 18, 2019

Flashed Blitzwolf BW-SHP6 using rpi3b+ on stretch without any issues!

@fenio
Copy link

fenio commented Dec 22, 2019

I've just flashed two SmartDGM PP-W162 plugs using master branch without issues.
Well it died during flashing second plug but retry did the trick. It was probably waiting too short.
Thanks for fabulous work.

Tested on Debian testing/unstable.

@BigGeorgeTx
Copy link

BigGeorgeTx commented Jul 25, 2020

smarthack-udp.log
Failed flashing Teckin SB50 bulb.

I have previously successfully flashed numerous wall switches, smart plugs and bulbs from various manufacturers using Tuya firmware. This is the first one I haven't been able to flash. It came with the 1.0.5 firmware that I have read has caused problems for many users. I am using the latest Tuya-convert software (2.2.4 ) on a Pi 4B. Bulb flashes fast, then after 1st step stops flashing, but the second step never completes. It keeps retrying until it times out.

Looking at the logs, what seemed most interesting was in smarthack-psk.log:
Traceback (most recent call last):
file "./psk-frontend.py", line 6, in
import sslpsk
ModuleNotFoundError: No module named 'sslpsk'
Smarthack-psk.log (END)

Searching on that error I found a suggestion to install
libssl-dev, which I tried with sudo apt-get install libssl-dev. The response from my Pi was that I already had the latest version installed (1.1.id-0+deb10u3+rpt1)

Does this just mean that I'm running up against the new firmware limits, or are there additional steps I can take?

Here are the log files:

smarthack-udp.log
smarthack-web.log
smarthack-wifi.log
smarthack-psk.log

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet