Solution to some of the Xiaomi Gateway timeouts

[sonartribe]

I've posted this in #20329
For completeness...

I have a theory...One thing I have noticed, and including myself, is that the people who appear to have most problems all have MAC address of the gateway starting 04:XX:XX:XX:XX:XX.

When I get updates from a sensor, I can see in the debug these appearing as a PUSH and the sensor gets updated.

2019-02-24 20:24:33 DEBUG (Thread-23) [xiaomi_gateway] MCAST (heartbeat) << {'model': 'sensor_ht', 'sid': '158dXXXXXXXXXX', 'cmd': 'heartbeat', 'short_id': 63981, 'data': '{"voltage":2985,"
temperature":"1845","humidity":"5269"}'}
2019-02-24 20:24:33 DEBUG (MainThread) [homeassistant.components.xiaomi_aqara] PUSH >> <Entity Humidity_158dXXXXXXXXXX: 52.7>: {'voltage': 2985, 'temperature': '1845', 'humidity': '5269'}
2019-02-24 20:24:33 DEBUG (MainThread) [homeassistant.components.xiaomi_aqara] PUSH >> <Entity Temperature_158dXXXXXXXXXX: 18.4>: {'voltage': 2985, 'temperature': '1845', 'humidity': '5269'
}

However when I get an illumination update from the gateway I don't get the corresponding PUSH debug message.

2019-02-24 20:20:27 DEBUG (Thread-23) [xiaomi_gateway] MCAST (report) << {'model': 'gateway', 'sid': '04XXXXXXXXXX', 'cmd': 'report', 'short_id': 0, 'data': '{"rgb":1694434079,"illumination
":336}'}

One thing I notice that the entity_id that home assistant set up, is

sensor.illumination_4XXXXXXXXXX
and
light.gateway_light_4XXXXXXXXXX

thus missing the leading zero.

Could this be causing a mismatch between the SID being broadcast and the HA entitiy?

Okay. Bit more digging, looks like during the discovery phase, the gateway drops this 0 in its response as seen in this nmap trace...

U 192.168.1.5:33094 -> 192.168.1.71:9898
{"cmd":"read","sid":"04xxxxxxxxxx"}

U 192.168.1.71:9898 -> 192.168.1.5:33094
{"cmd":"read_ack","model":"gateway","sid":"4xxxxxxxxxx","short_id":0,"data":"{"rgb":1694498815,"illumination":520,"proto_version":"1.1.2"
}"}

To work around this issue, I modified the xiaomi_gateway init.py in /srv/homeassistant/lib/python3.5/site-packages/xiaomi_gateway on my system. In the following snippet I changed

"sid": resp["sid"],
to
"sid": resp["sid"].rjust(12, '0'),

to pad an extra 0 back to the start of the SID.

        for device_type in device_types:
            if model in device_types[device_type]:
                supported = True
                xiaomi_device = {
                    "model": model,
                    "proto": self.proto,
                    "sid": resp["sid"].rjust(12, '0'),
                    "short_id": resp["short_id"] if "short_id" in resp else 0,
                    "data": _list2map(_get_value(resp)),
                    "raw_data": resp}
                self.devices[device_type].append(xiaomi_device)

now I see the PUSH messages, which according to the code will also reset the last seen item and hence stop the timeouts.

2019-02-24 21:52:09 DEBUG (Thread-23) [xiaomi_gateway] MCAST (report) << {'sid': '04xxxxxxxxxx', 'model': 'gateway', 'cmd': 'report', 'data': '{"rgb":0,"illumination":503}', 'short_id': 0}
2019-02-24 21:52:09 DEBUG (MainThread) [homeassistant.components.xiaomi_aqara] PUSH >> <Entity Illumination_04xxxxxxxxxx: 223>: {'rgb': 0, 'illumination': 503}
2019-02-24 21:52:09 DEBUG (MainThread) [homeassistant.components.xiaomi_aqara] PUSH >> : {'rgb': 0, 'illumination': 503}

Can somebody who know what they're doing add this into the codebase.

[scott-ip1]

Keen to see how this works out. My gateways both start with MAC of 04, and I see the same dropped leading zero and the 2.5hour unknown issue with lights not controlled properly.

[sonartribe]

Mine's been stable for 19 hours since I made this change, and I get updates from the switch and light sensors.

[amelchio]

CC @syssi

[syssi]

Great catch! I will prepare a PR.

[scott-ip1]

Maybe a noob question, but is there a way to know when this fix will appear in an HA update, or should we apply it manually for now?

[amelchio]

It is in 0.88.2 and also the 0.89 beta.

[scott-ip1]

Awesome. Updated and it's looking good!