Add OTA file for Xiaomi `MCCGQ14LM` #138

Otnow · 2022-07-16T13:11:13Z

"releaseDate": 1645770597000,
"part": "zigbee",
"downloadUrl": "https://cdn.aqara.com/cdn/opencloud-product/mainland/product-firmware/prd/lumi.magnet.acn001/20220223110724_OTA_lumi.magnet.acn001_0.0.0_0008_20220113212339_66ff9a.ota",
"upgradeTakesTime": 0,
"updateLog": "Device stability was increased.",
"updateTime": 1645770597000,
"isNecessary": 0,
"fileSize": 160352,
"createTime": 1645585938000,
"firmwareMD5": "25bb573f50d3ad22d2c5886adb66ff9a",
"state": 3,
"isForce": false,
"firmwareVersion": "0.0.0_0008",
"imageType": "exe",
"firmwareCRC32": "579D35D3"

TheJulianJES · 2022-07-16T18:36:33Z

Were you able to successfully update the E1 door sensors with this image? Using zigpy, I tried to parse this image and it says that the data is too short. It could be that Xiaomi has decided to use some special OTA format again, hence the question if you got it to work.
(On the other hand, the image for the P1 motion sensors parses just fine. I have yet to update mine though)
(And thanks for getting all the images!)

Edit: Image doesn't seem to be a proper Zigbee OTA file: Firmware does not start with expected magic bytes: b'\x00\xf0 r\x02\x00\x01\x0c\xd5\x8b'

puddly · 2022-07-16T19:35:51Z

It's only a little broken:

In [1]: import pathlib

In [2]: import zigpy.ota.image

In [3]: data = pathlib.Path('/tmp/20220223110724_OTA_lumi.magnet.acn001_0.0.0_0008_20220113212339_66ff9a.ota').read_bytes()

# Valid header
In [4]: hdr, data = zigpy.ota.image.OTAImageHeader.deserialize(data)

In [5]: hdr
Out[5]: OTAImageHeader(upgrade_file_id=200208670, header_version=256, header_length=56, field_control=<FieldControl.0: 0>, manufacturer_id=4447, image_type=10635, file_version=8, stack_version=2, header_string='Telink OTA Sample Usage', image_size=160352, *device_specific_file=False, *hardware_versions_present=False, *key=ImageKey(manufacturer_id=4447, image_type=10635), *security_credential_version_present=False)

# Has a single subelement
In [6]: subelement, data = zigpy.ota.image.SubElement.deserialize(data)

In [7]: subelement.tag_id
Out[7]: <ElementTagId.undefined_0xf000: 61440>

# But then has two bogus bytes at the end
In [8]: data
Out[8]: b'g\xf6'

I'm not sure if the subelement has an incorrect length and these two bytes are actually a part of it, or if they're trailing data at the end of the image.

Otnow · 2022-07-16T23:01:31Z

Were you able to successfully update the E1 door sensors with this image? Using zigpy, I tried to parse this image and it says that the data is too short.

As in the case of the P1 motion sensor, I managed to get a link to the OTA file for the E1 door sensor without physically having them, so, unfortunately, I can not check the firmware update process.

It could be that Xiaomi has decided to use some special OTA format again, hence the question if you got it to work.

It is quite possible, but in my memory this is the first such case with OTA files for Xiaomi devices.

(And thanks for getting all the images!)

You are welcome 🙂

I'm not sure if the subelement has an incorrect length and these two bytes are actually a part of it, or if they're trailing data at the end of the image.

Is it possible to fix this OTA file so that it is suitable for updating like the rest?

Or should I close this PR due to the unsuitability of the OTA file for updating?

puddly · 2022-07-16T23:18:00Z

An OTA image doesn't necessarily have to be spec-compliant for it to be accepted by a device (unfortunately), though I am curious to know if the gateway parses and re-wraps this image before sending it, or if it is indeed sent as-is. Without a traffic capture it's hard to tell.

That being said, none of the other ~50 Xiaomi/Aqara OTA images fail to parse like this. The only other one with this non-standard 0xf000 subelement is for lumi.switch.n2acn1, but it contains a firmware upgrade section and is otherwise well-formed:

/tmp/ota/OTA_lumi.switch.n2acn1_0.0.0_1548_20211125_D550CC.ota
Header: OTAImageHeader(upgrade_file_id=200208670, header_version=256, header_length=56, field_control=<FieldControl.0: 0>, manufacturer_id=4447, image_type=2700, file_version=3888, stack_version=2, header_string='ROUTERX-----JN5180--ENCRYPTED000', image_size=300232, *device_specific_file=False, *hardware_versions_present=False, *key=ImageKey(manufacturer_id=4447, image_type=2700), *security_credential_version_present=False)
Subelements: {<ElementTagId.UPGRADE_IMAGE: 0>: 281664, <ElementTagId.undefined_0xf007: 61447>: 128, <ElementTagId.undefined_0xf000: 61440>: 18232, <ElementTagId.undefined_0xf006: 61446>: 128}
Validation result: ValidationResult.UNKNOWN

Otnow · 2022-07-17T01:16:20Z

An OTA image doesn't necessarily have to be spec-compliant for it to be accepted by a device (unfortunately), though I am curious to know if the gateway parses and re-wraps this image before sending it, or if it is indeed sent as-is. Without a traffic capture it's hard to tell.

Ok. So someone should try to update their sensor with this OTA file or is it not possible due to validation failure?

The only other one with this non-standard 0xf000 subelement is for lumi.switch.n2acn1, but it contains a firmware upgrade section and is otherwise well-formed:

By the way, there is a new firmware version for lumi.switch.n2acn1: #142

Is it also with a non-standard structure?

TheJulianJES · 2022-07-17T04:19:24Z

Well, just tried installing the OTA update (unmodified) on an E1 door/window sensor and it didn't work (I just removed check in zigpy). The upgrade process just restarted at the end (and the battery is pretty empty now lol).

I guess I can try to send a re-wrapped image or one without the last two bytes at the end. Just sending "as is" doesn't seem to work, so I guess the sensor must do some validation?

(Btw., the P1 motion sensor upgrade worked flawlessly.)

TheJulianJES · 2022-07-17T04:29:03Z

@puddly If you're still around and have some time, could you modify the image to either re-wrap it and/or just remove the two bytes at the end (so I can test those images)?

puddly · 2022-07-17T04:38:14Z

If you follow the code example I posted above, you can generate two possible images:

subelement.data += b'those two trailing bytes (add the trailing bytes to the subelement)
hdr.image_size -= 2 (remove them from the image entirely)

In both cases, new_image = hdr.serialize() + subelement.serialize() can be written to a file (pathlib.Path('image.ota').write_bytes(new_image)) and should parse with the zigpy ota info utility.

TheJulianJES · 2022-07-17T05:15:30Z

Thanks for the steps!
I tried the re-wrapped image and it completed "successfully" but left the sensor bricked :(
Even tried a new (tested) battery but no signs of life.
Tbh, I don't want to risk another sensor atm (basically can't get them at the moment here) (unless it's very certain that the just removing those two bytes is the way to go)

(Looks like it has a TLSR8258 F512ET32 chip inside)

Otnow · 2022-07-17T12:01:02Z

@TheJulianJES, thanks for testing this OTA file.
It's a pity that your sensor is broken. I assumed that the device would validate any new firmware, but that doesn't seem to be the case.

@puddly, thanks for the help.

I close the PR, because in fact, it turned out that this OTA file is not suitable for updating using the standard procedure.

TheJulianJES · 2023-02-09T16:29:11Z

@Otnow I've received some new E1 door sensors (manufactured 10/22) and they shipped with firmware version 9.
If you have some time, can you check if there's a newer update available than this broken v8 update?

Otnow · 2023-02-09T17:38:35Z

@Otnow I've received some new E1 door sensors (manufactured 10/22) and they shipped with firmware version 9.
If you have some time, can you check if there's a newer update available than this broken v8 update?

Yes, there is such firmware:

"releaseDate": 1667875397000,
"part": "zigbee",
"downloadUrl": "https://cdn.aqara.com/cdn/opencloud-product/mainland/product-firmware/prd/lumi.magnet.acn001/20221105152917_OTA_lumi.magnet.acn001_0.0.0_0009_20221103_481485.ota",
"upgradeTakesTime": 0,
"updateLog": "Fixes known issues.",
"updateTime": 1667875397000,
"isNecessary": 0,
"fileSize": 163136,
"createTime": 1667875397000,
"firmwareMD5": "9b168b81d889169deb6fb65091481485",
"state": 2,
"isForce": false,
"firmwareVersion": "0.0.0_0009",
"imageType": "exe",
"firmwareCRC32": "408DA936"

, but, unfortunately, it, like the previous version, does not pass the test 😕:

> zigpy ota info 20221105152917_OTA_lumi.magnet.acn001_0.0.0_0009_20221103_481485.ota

WARNING Failed to parse 20221105152917_OTA_lumi.magnet.acn001_0.0.0_0009_20221103_481485.ota: Data is too short

puddly · 2023-02-09T17:46:12Z

It's "broken" in exactly the same way as the first one:

In [1]: import pathlib

In [2]: import zigpy.ota.image

In [3]: data = pathlib.Path('~/Downloads/20221105152917_OTA_lumi.magnet.acn001_0.0.0_0009_20221103_481485.ota').expanduser().read_bytes()

In [4]: hdr, data = zigpy.ota.image.OTAImageHeader.deserialize(data)

In [5]: hdr
Out[5]: OTAImageHeader(upgrade_file_id=200208670, header_version=256, header_length=56, field_control=<FieldControl.0: 0>, manufacturer_id=4447, image_type=10635, file_version=9, stack_version=2, header_string='Telink OTA Sample Usage', image_size=163136, *device_specific_file=False, *hardware_versions_present=False, *key=ImageKey(manufacturer_id=4447, image_type=10635), *security_credential_version_present=False)

In [6]: subelement, data = zigpy.ota.image.SubElement.deserialize(data)

In [7]: data
Out[7]: b'\xa7h'

In [8]: subelement.tag_id
Out[8]: <ElementTagId.undefined_0xf000: 61440>

The images aren't padded to 64 bytes so I'm not sure what these two trailing bytes could be.

@TheJulianJES tested one of the possible re-encodings of the OTA file (was it 1? or 2?) so I guess that just leaves the other one to try 😅

TheJulianJES · 2023-02-09T19:38:04Z

Tested the v9 image with removing the two extra bytes at the end now (my first try was with the rewrapped image and v8).
... but that still bricked the sensor.

puddly · 2023-02-09T19:47:08Z

I think we would need to capture the Aqara gateway updating one of these things, unfortunately. It has to do some transformation to make the file usable.

TheJulianJES · 2023-02-21T21:51:41Z

@Otnow If possible, can you post the links of all firmware versions (including the older ones (especially v5)) for the E1 door sensor?

I temporarily have an Aqara Hub M2 now (set to China Mainland servers).
The E1 door sensor is on 0.0.0_0004 and the hub is updating it to 0.0.0_0005 right now (so not the latest version). Perhaps the updates have to be done in steps. I'm trying to also sniff the update (between the sensor and hub, but there are two CRC errors already...)

Edit:
0.0.0_0004 to 0.0.0_0005 ✅ (capture has 5 CRC errors though)
0.0.0_0005 to 0.0.0_0009 ✅ (capture has 0 CRC errors -- repositioned hub/sniffer)

From a quick look comparing the v5 to v9 packet capture to the v9 Aqara OTA link, the last two bytes b\xa7h are sent.

TheJulianJES · 2023-02-21T22:57:22Z

These are all the OTA versions apparently:
(still missing the links though)

[
  {
    "releaseTime": "1671070353000",
    "necessary": 0,
    "fileSize": 163136,
    "updateLog": "Fixes known issues.",
    "state": 2,
    "firmwareVersion": "0.0.0_0009"
  },
  {
    "releaseTime": "1645770597000",
    "necessary": 0,
    "fileSize": 160352,
    "updateLog": "Device stability was increased.",
    "state": 2,
    "firmwareVersion": "0.0.0_0008"
  },
  {
    "releaseTime": "1642768319000",
    "necessary": 0,
    "fileSize": 158928,
    "updateLog": "Optimize the user experience",
    "state": 2,
    "firmwareVersion": "0.0.0_0007"
  },
  {
    "releaseTime": "1634550910000",
    "necessary": 1,
    "fileSize": 157424,
    "updateLog": "Optimize the user experience",
    "state": 2,
    "firmwareVersion": "0.0.0_0005"
  }
]

Otnow · 2023-02-21T23:15:33Z

Yes, there are links to versions 5, 8 and 9, except for 7:

"releaseDate": 1634550910000,
"part": "zigbee",
"downloadUrl": "https://cdn.aqara.com/cdn/opencloud-product/mainland/product-firmware/prd/lumi.magnet.acn001/20211016100134_OTA_lumi.magnet.acn001_0.0.0_0005_20211015230808_f3db8b.ota",
"upgradeTakesTime": 0,
"updateLog": "Optimize the user experience",
"updateTime": 1634550910000,
"isNecessary": 1,
"fileSize": 157424,
"createTime": 1634550910000,
"firmwareMD5": "98fb2c5bc736a5260b48547dbff3db8b",
"state": 3,
"isForce": false,
"firmwareVersion": "0.0.0_0005",
"imageType": "exe",
"firmwareCRC32": "E409417D"

"releaseDate": 1645770597000,
"part": "zigbee",
"downloadUrl": "https://cdn.aqara.com/cdn/opencloud-product/mainland/product-firmware/prd/lumi.magnet.acn001/20220223110724_OTA_lumi.magnet.acn001_0.0.0_0008_20220113212339_66ff9a.ota",
"upgradeTakesTime": 0,
"updateLog": "Device stability was increased.",
"updateTime": 1645770597000,
"isNecessary": 0,
"fileSize": 160352,
"createTime": 1645585938000,
"firmwareMD5": "25bb573f50d3ad22d2c5886adb66ff9a",
"state": 3,
"isForce": false,
"firmwareVersion": "0.0.0_0008",
"imageType": "exe",
"firmwareCRC32": "579D35D3"

"releaseDate": 1667875397000,
"part": "zigbee",
"downloadUrl": "https://cdn.aqara.com/cdn/opencloud-product/mainland/product-firmware/prd/lumi.magnet.acn001/20221105152917_OTA_lumi.magnet.acn001_0.0.0_0009_20221103_481485.ota",
"upgradeTakesTime": 0,
"updateLog": "Fixes known issues.",
"updateTime": 1667875397000,
"isNecessary": 0,
"fileSize": 163136,
"createTime": 1667875397000,
"firmwareMD5": "9b168b81d889169deb6fb65091481485",
"state": 2,
"isForce": false,
"firmwareVersion": "0.0.0_0009",
"imageType": "exe",
"firmwareCRC32": "408DA936"

Add OTA file for Xiaomi MCCGQ14LM

a1a2f1c

Otnow closed this Jul 17, 2022

Otnow deleted the Aqara-MCCGQ14LM branch July 17, 2022 12:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add OTA file for Xiaomi `MCCGQ14LM` #138

Add OTA file for Xiaomi `MCCGQ14LM` #138

Otnow commented Jul 16, 2022

TheJulianJES commented Jul 16, 2022 •

edited

Loading

puddly commented Jul 16, 2022

Otnow commented Jul 16, 2022

puddly commented Jul 16, 2022

Otnow commented Jul 17, 2022

TheJulianJES commented Jul 17, 2022 •

edited

Loading

TheJulianJES commented Jul 17, 2022

puddly commented Jul 17, 2022 •

edited

Loading

TheJulianJES commented Jul 17, 2022 •

edited

Loading

Otnow commented Jul 17, 2022

TheJulianJES commented Feb 9, 2023

Otnow commented Feb 9, 2023

puddly commented Feb 9, 2023

TheJulianJES commented Feb 9, 2023

puddly commented Feb 9, 2023

TheJulianJES commented Feb 21, 2023 •

edited

Loading

TheJulianJES commented Feb 21, 2023

Otnow commented Feb 21, 2023

Add OTA file for Xiaomi MCCGQ14LM #138

Add OTA file for Xiaomi MCCGQ14LM #138

Conversation

Otnow commented Jul 16, 2022

TheJulianJES commented Jul 16, 2022 • edited Loading

puddly commented Jul 16, 2022

Otnow commented Jul 16, 2022

puddly commented Jul 16, 2022

Otnow commented Jul 17, 2022

TheJulianJES commented Jul 17, 2022 • edited Loading

TheJulianJES commented Jul 17, 2022

puddly commented Jul 17, 2022 • edited Loading

TheJulianJES commented Jul 17, 2022 • edited Loading

Otnow commented Jul 17, 2022

TheJulianJES commented Feb 9, 2023

Otnow commented Feb 9, 2023

puddly commented Feb 9, 2023

TheJulianJES commented Feb 9, 2023

puddly commented Feb 9, 2023

TheJulianJES commented Feb 21, 2023 • edited Loading

TheJulianJES commented Feb 21, 2023

Otnow commented Feb 21, 2023

Add OTA file for Xiaomi `MCCGQ14LM` #138

Add OTA file for Xiaomi `MCCGQ14LM` #138

TheJulianJES commented Jul 16, 2022 •

edited

Loading

TheJulianJES commented Jul 17, 2022 •

edited

Loading

puddly commented Jul 17, 2022 •

edited

Loading

TheJulianJES commented Jul 17, 2022 •

edited

Loading

TheJulianJES commented Feb 21, 2023 •

edited

Loading