Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for pronto codes on chuangmi.remote.h102a03 #495

Open
nicole-ashley opened this issue Apr 9, 2019 · 63 comments · May be fixed by #1021
Open

Support for pronto codes on chuangmi.remote.h102a03 #495

nicole-ashley opened this issue Apr 9, 2019 · 63 comments · May be fixed by #1021

Comments

@nicole-ashley
Copy link

nicole-ashley commented Apr 9, 2019

I have a model of the Xiaomi IR remote that reports as chuangmi.remote.h102a03. It works fine when capturing and replaying those captured raw commands, but any commands converted from Pronto do nothing - no flashing light, no response from the target device, but - quite puzzlingly - an ok response from the device's API.

Here is the device on Ali Express.

Here are some examples:

>>> ir.info()
DEBUG:miio.device:10.1.1.23:54321 >>: {'id': 3, 'method': 'miIO.info', 'params': []}
DEBUG:miio.device:10.1.1.23:54321 >>: {'id': 104, 'method': 'miIO.info', 'params': []}
DEBUG:miio.device:10.1.1.23:54321 (ts: 1970-01-30 15:52:36, id: 104) << {'result':
  {'life': 2562756, 'cfg_time': 0, 'token': 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx', 
   'mac': 'xx:xx:xx:xx:xx:xx', 'fw_ver': '1.4.3_10', 'hw_ver': 'MC200', 
   'model': 'chuangmi.remote.h102a03', 'wifi_fw_ver': 'SD878x-14.76.36.p79-702.1.0-WM', 
   'ap': {'rssi': -63, 'ssid': 'xxxxxxxxx', 'bssid': 'xx:xx:xx:xx:xx:xx'}, 
   'netif': {'localIp': '10.1.1.23', 'mask': '255.255.255.0', 'gw': '10.1.1.1', 
             'gw_mac': 'xx:xx:xx:xx:xx:xx' },
  'mmfree': 56428, 'ot': 'otu', 'otu_stat': [325, 333, 3647, 125, 3134, 10], 
  'ott_stat': [19, 4, 3, 612]}, 'id': 104}
chuangmi.remote.h102a03 v1.4.3_10 (xx:xx:xx:xx:xx:xx) @ 10.1.1.23 - token: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx


# Works (previously recorded command):

>>> ir.play_raw('tllsNyt1am1Wp1Bo1ZodBoNDtVCoNBoNRoNrAFKtWcAQgAPAA8ADwAPBTKi0IFAgViooB53eg2cAsqiAWQCaUGg02g0kAsgIyAfOogZpPQ==')
DEBUG:miio.device:10.1.1.23:54321 >>: {'id': 2, 'method': 'miIO.ir_play', 
  'params': {'freq': 38400, 'code': 'tllsNyt1am1Wp1Bo1ZodBoNDtVCoNBoNRoNrAFKtWcAQgAPAA8ADwAPBTKi0IFAgViooB53eg2cAsqiAWQCaUGg02g0kAsgIyAfOogZpPQ=='}}
DEBUG:miio.device:10.1.1.23:54321 (ts: 1970-01-30 15:49:27, id: 2) << {'result': ['ok'], 'id': 2}
['ok']


# Doesn't work (Pronto code that should do the same):

>>> ir.play_pronto('0000 0067 0000 000D 0060 0018 0018 0018 0030 0018 0030 0018 0030 0018 0018 0018 0030 0018 0018 0018 0030 0018 0018 0018 0018 0018 0018 0018 0018 03F6')
DEBUG:miio.device:10.1.1.23:54321 >>: {'id': 419, 'method': 'miIO.ir_play', 
  'params': {'freq': 40244, 'code': 'Z6UzAFQCAACoBAAAUQkAAGxiAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAEBAQABAAEAAAAwAgABAQEAAQABAAAAMA=='}}
DEBUG:miio.device:10.1.1.23:54321 (ts: 1970-01-30 16:28:16, id: 419) << {'result': ['ok'], 'id': 419}
['ok']

>>> ir.play_raw('6UzAFQCAACoBAAAUQkAAGxiAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAEBAQABAAEAAAAwAgABAQEAAQABAAAAMA==', 40244)
DEBUG:miio.device:10.1.1.23:54321 >>: {'id': 420, 'method': 'miIO.ir_play', 
  'params': {'freq': 40244, 'code': '6UzAFQCAACoBAAAUQkAAGxiAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAEBAQABAAEAAAAwAgABAQEAAQABAAAAMA=='}}
DEBUG:miio.device:10.1.1.23:54321 (ts: 1970-01-30 16:28:39, id: 420) << {'result': ['ok'], 'id': 420}
['ok']


# From specs (also does nothing):

>>> ir.play_pronto('0000 006C 0022 0002 015B 00AD 0016 0016 0016 0016 0016 0016 0016 0016 0016 0016 0016 0016 0016 0016 0016 0016 0016 0041 0016 0041 0016 0041 0016 0041 0016 0041 0016 0041 0016 0041 0016 0016 0016 0016 0016 0041 0016 0016 0016 0041 0016 0016 0016 0016 0016 0016 0016 0016 0016 0041 0016 0016 0016 0041 0016 0016 0016 0041 0016 0041 0016 0041 0016 0041 0016 0622 015B 0057 0016 0E6C')
DEBUG:miio.device:10.1.1.23:54321 >>: {'id': 524, 'method': 'miIO.ir_play', 
  'params': {'freq': 38381, 'code': 'Z6VHAD0CAACdBgAA2ggAAJsRAABQIwAAyZ8AAMF3AQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA0AAAAAAAAAAAQEBAQEBAQAAAQABAAAAAAEAAQABAQEBBQJGA='}}
DEBUG:miio.device:10.1.1.23:54321 (ts: 1970-01-30 16:42:56, id: 524) << {'result': ['ok'], 'id': 524}
['ok']

>>> ir.play_raw('Z6VHAD0CAACdBgAA2ggAAJsRAABQIwAAyZ8AAMF3AQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA0AAAAAAAAAAAQEBAQEBAQAAAQABAAAAAAEAAQABAQEBBQJGA=', 38381)
DEBUG:miio.device:10.1.1.23:54321 >>: {'id': 526, 'method': 'miIO.ir_play', 
  'params': {'freq': 38381, 'code': 'Z6VHAD0CAACdBgAA2ggAAJsRAABQIwAAyZ8AAMF3AQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA0AAAAAAAAAAAQEBAQEBAQAAAQABAAAAAAEAAQABAQEBBQJGA='}}
DEBUG:miio.device:10.1.1.23:54321 (ts: 1970-01-30 16:43:58, id: 526) << {'result': ['ok'], 'id': 526}
['ok']

Unfortunately I don't have a clue where to start, or I'd tinker with the code myself. I'm happy to work with you if you need to test things on my device.

@syssi
Copy link
Collaborator

syssi commented Apr 10, 2019

Could you try to play pronto codes with 38400 baud? Could you provide the mDNS name of the device? I would like to add it to the list of supported/discoverable devices:

https://github.com/rytilahti/python-miio/blob/master/miio/discovery.py#L53

@nicole-ashley
Copy link
Author

nicole-ashley commented Apr 10, 2019

Thanks for your quick response!

I assume you mean a raw converted from the Pronto code? The Pronto method doesn't have a baud/frequency parameter.

# My code 
>>> ir.play_raw('6UzAFQCAACoBAAAUQkAAGxiAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAEBAQABAAEAAAAwAgABAQEAAQABAAAAMA==', 38400)
['ok']

# From specs
>>> ir.play_raw('Z6VHAD0CAACdBgAA2ggAAJsRAABQIwAAyZ8AAMF3AQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA0AAAAAAAAAAAQEBAQEBAQAAAQABAAAAAAEAAQABAQEBBQJGA=', 38400)
['ok']

Same thing - no flashing, no transmission.

The mDNS name as far as I can tell is chuangmi-remote-h102a03_miio108732461, so I'm guessing a good identifier would be chuangmi-remote-h102a03_.

EDIT: I realise I wasn't in debug mode for these two tests - let me know if you want me to run them again with debug logging on.

@rezmus
Copy link

rezmus commented Apr 12, 2019

following methods are also present in firmware:

get_indicatorLamp
set_indicatorLamp
ir_play_batch

i can share firmware file in case you are interested for analyze.

@nicole-ashley
Copy link
Author

get_indicatorLamp and set_indicatorLamp probably refer to the option in the Xiaomi app to disable the blue LED on the front.

@syssi
Copy link
Collaborator

syssi commented Apr 13, 2019

@yawor Do you like to provide some support here? It's about pronto codes ;-)

@syssi
Copy link
Collaborator

syssi commented Apr 13, 2019

It looks like the LED of the devices can be controlled by:

# current status
miiocli device --ip IP --token TOKEN raw_command get_indicatorLamp "[]"

# Disable the indicator LED and query the current status
miiocli device --ip IP --token TOKEN raw_command set_indicatorLamp "['off']"
miiocli device --ip IP --token TOKEN raw_command get_indicatorLamp "[]"

# Enabled the indicator LED and query the current status
miiocli device --ip IP --token TOKEN raw_command set_indicatorLamp "['on']"
miiocli device --ip IP --token TOKEN raw_command get_indicatorLamp "[]"

If somebody likes to test these commands I will extend python-miio afterwards.

@rezmus
Copy link

rezmus commented Apr 13, 2019

"chuangmi-remote-h102a03_": ChuangmiIr,

you should remove underscore.

@syssi
Copy link
Collaborator

syssi commented Apr 13, 2019

I used the underscore as separator here.

@rezmus
Copy link

rezmus commented Apr 13, 2019

for all miio wifi devices you can get model name like this:

$model = str_replace("-",".",stristr($ssid,"_",true));

so underscore should be redundant here.

@syssi
Copy link
Collaborator

syssi commented Apr 13, 2019

If another model is called "chuangmi-remote-h102a03b" and incompatible with the ChuangmiIr implementation it shouldn't be reported as supported because "chuangmi-remote-h102a03" is matching. The underscore does a good job in this case.

@rezmus
Copy link

rezmus commented Apr 13, 2019

OK, but with this logic you might as well add underscore to any device listed in discovery.py, because if they release for example zhimi-airpurifier-v10 (match zhimi-airpurifier-v1) it shouldn't be reported as supported ;) or model matching should be done like i wrote above, it's how miio sdk does it and how mi home builds model string to compare with model list for selected region.

anyway it's fine for me, one underscore is not worth an argue ;)

@syssi
Copy link
Collaborator

syssi commented Apr 13, 2019

You are totally right. Any input is appreciated! :-) Feel free to provide additional feedback and/or PRs. :-)

@nicole-ashley
Copy link
Author

@syssi My suspicions were correct, this is the blue LED on the front which is usually on all the time and flashes when a command is sent. It's the light I'm referring to "not flashing" when the codes above are not understood. If you turn it off, it's off all the time even when successfully playing commands.

This corresponds to the setting in the app called "Instruction light", but I think "Indicator light" is a more accurate English name.

Screenshot_20190414-101101~2
IMG_20190414_102814~2

$ miiocli device raw_command get_indicatorLamp "[]"
Running command raw_command
['on']
$ miiocli device raw_command set_indicatorLamp "['off']"
Running command raw_command
['ok']
$ miiocli device raw_command get_indicatorLamp "[]"
Running command raw_command
['off']

Screenshot_20190414-101744~2
IMG_20190414_102845~2

$ miiocli device raw_command set_indicatorLamp "['on']"
Running command raw_command
['ok']
$ miiocli device raw_command get_indicatorLamp "[]"
Running command raw_command
['on']

Screenshot_20190414-101101~2
IMG_20190414_102814~2

@yawor
Copy link
Contributor

yawor commented Apr 14, 2019

@syssi this is not definitive as I've only glimpsed on what @nikrolls posted, but it seems that the IR format has changed. If I'm correct then this needs to be reverse engineered like the previous format for chuangmi.
For that I need some reference signal to be able to understand the new format.
@nikrolls can you share more info regarding the equipment for which you've recorded the signal that works (the one you've posted in opening post)? Manufacturer, device type and model, maybe even which function on the remote you've used to record that signal.
--edit
Upon further inspection, I'm certain that this is now a different format. Until now every signal (in raw binary form, not the base64 string) started with 0xA567. This one starts with 0xB659 (I'd need to see more recorded signals for this new format to know if this value is constant or not). The next two bytes should contain number of edges but it doesn't seem to be true for the posted signal as it would then need to have 14188 edges recorded 😮

@yawor
Copy link
Contributor

yawor commented Apr 14, 2019

@nikrolls do you have an universal remote control by any chance? If yes it would be helpful to program it to a device of a known protocol, parameters and commands.

@rezmus
Copy link

rezmus commented Apr 14, 2019

chuangmi.ir.v2 does not support led methods, chuangmi.remote.v2 does.

@nicole-ashley
Copy link
Author

@yawor These were recorded off the universal remote that I have - an Xbox One with Kinect. I can enter a 4-digit TV code and it will play discreet codes for power on and off. I'm testing on a Sony Brava 32V4000, but codes as low level as these are standard across most Sony TVs.

Raw codes below are captured using the Chuangmi H102A03, Pronto codes are according to an apparently reliable source on Remote Central.

# Power on (discreet code from Xbox Universal Remote)
Raw: tllsNyt1am1Wp1Bo1ZodBoNDtVCoNBoNRoNrAFKtWcAQgAPAA8ADwAPBTKi0IFAgViooB53eg2cAsqiAWQCaUGg02g0kAsgIyAfOogZpPQ==
Pronto: 0000 0067 0000 000D 0060 0018 0018 0018 0030 0018 0030 0018 0030 0018 0018 0018 0030 0018 0018 0018 0030 0018 0018 0018 0018 0018 0018 0018 0018 03F6

# Power off (discreet code from Xbox Universal Remote)
Raw: tllsNyt1am1Wp1Bo1nodBoNDqtCoNBoNVoNroNBptGsgBRAAeAB4AHgAfQ6DUaLQqDUaCCsVFAPO70GzgHmAWQCaAzFQaSAeYD61EDNJ6A==
Pronto: 0000 0067 0000 000D 0060 0018 0030 0018 0030 0018 0030 0018 0030 0018 0018 0018 0030 0018 0018 0018 0030 0018 0018 0018 0018 0018 0018 0018 0018 03DE

I also have other codes I captured from the TV's own remote:

# Volume up
Raw: tllsNyt1am1Wt1Bo1BodBoNDnNCoNBoNloNroNBplKtoBRAAeAB4AHgAeCmVFAGIAN6Dd6DZ6CDEYAb0Gm0Gk0Gg1EAggA3ogGf0Ggg=
Pronto: 0000 0067 0000 000D 0060 0018 0018 0018 0030 0018 0018 0018 0018 0018 0030 0018 0018 0018 0018 0018 0030 0018 0018 0018 0018 0018 0018 0018 0018 0426

# Volume down
Raw: tllsNyt1am1Wp1Bos0odBoNEoVCoNBoNkoNroNBotSAEKhXStAHkAB4AHgAeABVDoNRooAxABvQbvQbPRQYjADeg1Gg0mhACEAQYAa0Ggz0=
Pronto: 0000 0067 0000 000D 0060 0018 0030 0018 0030 0018 0018 0018 0018 0018 0030 0018 0018 0018 0018 0018 0030 0018 0018 0018 0018 0018 0018 0018 0018 040E

# Input select
Raw: tllsNyt1am1XsVBp1joVBoNBqlBs9BoNYs9JoNBp9jodBoNDooChzaogKFQ5xSqDQaFdq0AeVto9RAFIADwAPAAS20qiUG10m00m10mp1G00m00uhUq01G00qh0UAMgEzpNpsdqqoD5ANkA+QCZWik1ik20B8gM6o4GbUkDNQESs9prIGZ3cDNaLaaTZ6TPQ
Pronto: 0000 0067 0000 000D 0060 0018 0030 0018 0018 0018 0030 0018 0018 0018 0018 0018 0030 0018 0018 0018 0030 0018 0018 0018 0018 0018 0018 0018 0018 040E (I think)

# Arrow up
Raw: tllsNyt1am1XoVBoc0odBoNCsNBs9BoNtqNRoNBotrpVBoNBsNZAPIADwAPAA8ACqpRalQanRalSanRbRSajRalRqFRbxQajRbRRQDzAXKg1Ko1Km1IB8qkAelootSotFooGZAPlQgM0qVFoQGaAPkBm1CoNSoNRns9A
Pronto: 0000 0067 0000 000D 0060 0018 0030 0018 0018 0018 0018 0018 0018 0018 0030 0018 0030 0018 0018 0018 0030 0018 0018 0018 0018 0018 0018 0018 0018 040E

# Arrow down
Raw: tllsNyt1am1XsVBo13odBoNCrFCoNBoNvu9RoNBotZpVBoNEqNYu4CBAAeAB4AHgATRKDZ6CCuVJoVBs9JAOIF0ruAOVRATKz0UBcgCjpoD5gEFSaGAYQO5AZ+Bn4GdWcDNqDQZ6
Pronto: 0000 0067 0000 000D 0060 0018 0018 0018 0030 0018 0018 0018 0018 0018 0030 0018 0030 0018 0018 0018 0030 0018 0018 0018 0018 0018 0018 0018 0018 040E

# Enter
Raw: tllsNyt1am1XsVBolJodBoNCrFBs9BoNuu9RoNBolZpQB5VahgFFS6PSaCAB4AHgAeAA1VotooNTotSotTotopNSotSoVRot4AHK0AIlaKSAudSqtSqIA5AJlQgDyAXOrUUDP6lRqEBmdBAzmhU0DM6iBmoC5VGiz0A=
Pronto: 0000 0067 0000 000D 0060 0018 0030 0018 0030 0018 0018 0018 0030 0018 0018 0018 0018 0018 0018 0018 0030 0018 0018 0018 0018 0018 0018 0018 0018 040E

I should be able to set the Xbox to any other TV with a four digit code and capture at least power commands (and possibly volume commands) from it for reference.

@yawor
Copy link
Contributor

yawor commented Apr 14, 2019

@nikrolls awesome. This should be enough to attempt reversing the format if it's not encrypted or compressed in any complicated way.

@nicole-ashley
Copy link
Author

Great! Also let me know if you want me to set my Xbox to a specific model and capture power codes.

@yawor
Copy link
Contributor

yawor commented Apr 14, 2019

@nikrolls you could record few signals for example for LG and Samsung.
I've glimpsed over the vol up and vol down for Sony and they only differ on one position in Pronto, but the data from your chuangmi device is totally different. I know that they shouldn't be almost identical as recording a signal is a tricky business (the transmitter is usually not 100% stable and difference in clocks between transmitter and receiver - all this causes that the timings have some jitter), but these seem to have no obvious correlation with pronto signals. What's interesting is that first 9 bytes are identical. That's why I would like to see raw data of signals with different protocols to see if these are always the same or change depending on the signal in some way.
It's too late for me today, need to get some sleep. I'll try tomorrow.

@nicole-ashley
Copy link
Author

nicole-ashley commented Apr 14, 2019

Great - I'll record some from other manufacturers. I can also try recording the above signals again a few times to see if they change a lot every time. If they do, I wonder if there's some kind of hashing or encryption over a temporal value going on?

I probably won't get any recordings done in the next 24 hours, but I'll try after that.

@rezmus
Copy link

rezmus commented Apr 15, 2019

you should probably check libcompression.so from plugin lib folder, that's where decode/encode is done.

@yawor
Copy link
Contributor

yawor commented Apr 15, 2019

@rezmus that was awesome tip. It would take a lot longer if I had to do full analysis of the disassembled/decompiled code, but fortunately they've left names of functions which directed me to a heatshrink compression library. It is based on LZSS compression and has really low memory footprint so it's ideal for embedded applications like this.
I've cloned and built https://github.com/atomicobject/heatshrink and used the resulting binary to decompress binary data (after decoding base64 string) and I can confirm that this is it. The format is still not the same as before, but now I can work out the structure.
I've also found a python library (it's called heatshrink) which provides bindings to the above C library, but the problem is really poor Py3 compatibility. I've tried to use it with python versions from 3.5 to 3.7 and it fails on import. I'll try looking into the problem and maybe provide some PRs for them so we could use it.
--edit
Just to be sure I've installed the library under python 2.7 and it works properly and correctly decompresses the IR signal data.
--edit
Just for information, the heatshrink library uses cython to build native python extension. This means the target system needs to have a compiler and python-dev (header files) packages installed. I don't know if this is an issue for us, because python-miio already uses cryptography package, which also provides a native extension which needs to be built during installation.
--edit
There's already a pending PR with Py3 fixes in the pyheatshrink project but it's hard to tell when it's going to be merged. The last activity of the author was in December. We could just fork the repo, apply the PR and add dependency on that forked repository.

@yawor
Copy link
Contributor

yawor commented Apr 15, 2019

@rezmus BTW you can delete the link to the file. It's better not to keep it here longer than necessary (in case of some legal issues).

@yawor
Copy link
Contributor

yawor commented Apr 15, 2019

Good news! At the bottom level, the format is still the same as before. Here's what they did here:

  • The original raw signal format is base64 encoded (exactly like before)
  • The base64 string is prefixed with "learn" string (so it becomes "learnXXXXX" where XXXXX is the base64 encoded raw data)
  • The resulting string is compressed using Heatshrink and again base64 encoded.

So all we need to do is to add the 2 last steps to the current pronto conversion to add support for this device.

@nicole-ashley
Copy link
Author

Fantastic news! That was quick 😊

@yawor
Copy link
Contributor

yawor commented Apr 15, 2019

My proposition is to subclass ChuangmiIr class by creating ChuangmiRemote(ChuangmiIr) class. It would override the pronto_to_raw classmethod by applying extra steps. This way we don't need to change the API in any way. We can also move ChuangmiRemote specific methods (the indicator LED related ones) into the new class. Everything else can be inherited from the base class.

@yawor
Copy link
Contributor

yawor commented Apr 15, 2019

@nikrolls can you try these?
volume up:
tllsNyt1am1VvVBo1RodBoNDt9CoNBoNVqNroNBoVStwBRAAeAB4AHgAeCmVFAGIAN6Dd6DZ6CDEYAb0Gm0Gez0= with frequency set to 40244
volume down:
tllsNyt1am1VvVBo1RodBoNDt9CoNBoNVqNroNBptCAEIADwAPAA8AD6DQ6DUaKAMQAb0G70Gz0UGIwA3oNNoM9noA== with frequency also set to 40244

oblitum added a commit to oblitum/python-miio that referenced this issue Apr 22, 2021
oblitum added a commit to oblitum/python-miio that referenced this issue Apr 22, 2021
oblitum added a commit to oblitum/python-miio that referenced this issue Apr 22, 2021
oblitum added a commit to oblitum/python-miio that referenced this issue Apr 22, 2021
oblitum added a commit to oblitum/python-miio that referenced this issue Apr 22, 2021
oblitum added a commit to oblitum/python-miio that referenced this issue Apr 22, 2021
oblitum added a commit to oblitum/python-miio that referenced this issue Apr 22, 2021
oblitum added a commit to oblitum/python-miio that referenced this issue Apr 22, 2021
All kudos to original work by @yawor's on PR rytilahti#501.

Fixes rytilahti#495, fixes rytilahti#619, fixes rytilahti#811
Closes rytilahti#501
Partially covers rytilahti#1020
oblitum added a commit to oblitum/python-miio that referenced this issue Apr 22, 2021
All kudos to original work by @yawor's on PR rytilahti#501.

Fixes rytilahti#495, fixes rytilahti#619, fixes rytilahti#811
Closes rytilahti#501
Partially covers rytilahti#1020
oblitum added a commit to oblitum/python-miio that referenced this issue Apr 22, 2021
All kudos to original work by @yawor's on PR rytilahti#501.

Fixes rytilahti#495, fixes rytilahti#619, fixes rytilahti#811.
Closes rytilahti#501.
Partially covers rytilahti#1020.
oblitum added a commit to oblitum/python-miio that referenced this issue Apr 22, 2021
All kudos to original work by @yawor's on PR rytilahti#501.

Fixes rytilahti#495, fixes rytilahti#619, fixes rytilahti#811.
Closes rytilahti#501.
Partially covers rytilahti#1020.
oblitum added a commit to oblitum/python-miio that referenced this issue Apr 24, 2021
All kudos to original work by @yawor's on PR rytilahti#501.

Fixes rytilahti#495, fixes rytilahti#619, fixes rytilahti#811.
Closes rytilahti#501.
Partially covers rytilahti#1020.
@mpsOxygen
Copy link
Contributor

@rezmus could you give a link to the firmware pls?

@rezmus
Copy link

rezmus commented Jun 15, 2021

which model?

@mpsOxygen
Copy link
Contributor

chuangmi.remote.v2

@mpsOxygen
Copy link
Contributor

Thank you!

@mpsOxygen
Copy link
Contributor

Do you know of any tool that extracts the elf file from the .bin image? I'm seeing 7 segments and will probably end up dumping all of them, but I was just wondering if there is a ready made tool that does this already.

@yawor
Copy link
Contributor

yawor commented Jun 15, 2021

@mpsOxygen isn't it other way around? The firmware bin image should be a pure executable (just machine code + program data). ELF format defines a container for linkable execs (more or less) which is much more than what you have in the bin file. Microcontrollers usually don't use ELF, just pure machine code, as the entry point to the application is predetermined depending on the µC used.

@mpsOxygen
Copy link
Contributor

He explains it better than I can: https://www.youtube.com/watch?v=w4_3vwN_2dI

So the initial app is just an elf then that gets converted to esp32 binary format. They also wrote a tool, you can find it here: https://github.com/tenable/esp32_image_parser

Haven't used it yet.

@yawor
Copy link
Contributor

yawor commented Jun 15, 2021

OK, it's more like a conversion between formats than extraction.
The main issue is the lack of support for raw image format in the IDA Xtensa module. In addition to their tool there's also a custom IDA loader which is actually able to load raw image directly, without the manual conversion to ELF (it's mentioned briefly in the video at ~5:24):
https://github.com/jrozner/esp-image-ida

@mpsOxygen
Copy link
Contributor

I am using Ghidra and tried with this: https://github.com/tslater2006/esp32_flash_loader
But it did not work. The image parser does not work either. I am starting to run out of ideas...

@yawor
Copy link
Contributor

yawor commented Jun 16, 2021

Do you also have this module? https://github.com/yath/ghidra-xtensa
It's needed for Ghidra to understand ESP32 machine code.

@mpsOxygen
Copy link
Contributor

@yawor Yes I have the xtensa CPU extension added.

@rezmus Did you pull the firmware of the device or downloaded it from the cloud? I think this bin only contains a sort of diff to be updated and not the full firmware.

@yawor
Copy link
Contributor

yawor commented Jun 16, 2021

Can I also get the file?

@rezmus
Copy link

rezmus commented Jun 16, 2021

it's just ota, i don't have this model.

@yawor
Copy link
Contributor

yawor commented Jun 16, 2021

Thanks for the file. This a content of an application partition (doesn't matter if it's a factory, OTA1 or OTA2, the application format itself is the same). It means it's a complete executable running on the device, so it should be possible to disassemble it, but it's going to require some work. I'll look into it.

@yawor
Copy link
Contributor

yawor commented Jun 16, 2021

I've been able to convert the app image into ELF format after fixing the image2elf function a little (I've just had to add a single line) and calling it from another python file instead of using their command line parser and logic. The Ghidra loads the file and is able to analyse it.
The problem is that during the build process, when ESP-IDF generates bin image from compiled elf files, the symbol table is dropped, so function positions, variables, labels etc are lost. All you get is a lot of Xtensa assembler and few functions which Ghidra tried to decompile back into C.

@mpsOxygen
Copy link
Contributor

Looks like the yath Xtensa Processor for Ghidra is a bit lacking. Using this one I've had way better results: https://github.com/Ebiroll/ghidra-xtensa

Also use this svd loader plugin with the esp32.svd file in order to get all the peripherals: https://github.com/leveldown-security/SVD-Loader-Ghidra

Also using the C header files from esp-idf with Parse C source helps with some of the types.

Unfortunately there are so many functions and no names that it is really hard to pick out the ones that handle IR. :(

@yawor
Copy link
Contributor

yawor commented Jun 18, 2021

A lot of the code is from stdlib and ESP-IDF. Quite large part is the FreeRTOS code, which implement multi-threading. But recognising which parts of the binary contain what may be quite difficult task for sure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

10 participants