Checking Xiaomi Aqara devices unavailability states #11631

PaulAnnekov · 2018-01-13T23:21:30Z

Description:

This PR makes xiaomi aqara component track if each connected device is still available and update state accordingly.

Why is it needed?

Because any device connected to gateway is totally useless w/o this feature, especially such critical ones as leakage, gas, smoke, motion and door sensors. Why useless? Because they can switch off at any point of time because of bug/battery drain and you won't know about this. For days and months you will think they are still working, but they won't. You will understand they are not working, when you will come back from a holiday and see your house is a pool full of water :). I think I don't need further explanation of "Why?", I'm sure you will agree this check is a MUST for EVERY device connected to smart home.

How is it working

Each device is given 2.5 hours timeframe to send report/heartbeat or respond to read. This time is enough to get at least 2 heartbeats. When device shows it's alive making any of these actions - we reset the timer. If it's not - state is set to STATE_UNAVAILABLE. It will trigger state change + you will see the state in UI (screenshot below). When device shows it's alive again, we set state to None (like constructor does).

How to get notification when device becomes unavailable?

E.g. use automation to track unavailable state and send an email.

Average `heartbeat` time periods

magnet 1 hour
motion/sensor_motion.aq2 55 min
switch 1 hour
sensor_wleak.aq1 50 min
gateway 10 sec
sensor_ht 50 min - 1 hour
plug 7 min

Checklist:

Remove motion sensor ignores from PyXiaomiGateway.
The code change is tested and works locally.

@Danielhiversen this code Danielhiversen/PyXiaomiGateway@973a21d makes motion sensors switch to unavailable state because we ignore heartbeats. I see two issues with this commit:

You have written a general-purpose lib to talk to gateway, but you're ignoring essential part of gateway's API. I think it's not responsibility of your library to filter these messages. Users of library should decide what to do with them by themselves.
Is this issue still relevant? There were several firmware updates since that moment.

I suggest to remove this filter completely and:

Add a config option to the component to ignore heartbeats for motion sensors. That's in case users will find this issue is still relevant and want a fast fix. It's not easy task, because push_data receives data-only, it doesn't know if it's heartbeat or report, we will need to change PyXiaomiGateway.
Do nothing. There is a big chance this issue was fixed in the latest firmwares. If users will complain - we will implement 1st option.

What do you this about this?

…AILABLE now

…ILABLE and return it instead e.g. STATE_OFF

houndci-bot · 2018-01-13T23:21:42Z

homeassistant/components/xiaomi_aqara.py

-        if is_data or is_voltage:
-            self.schedule_update_ha_state()
+        # There is a chance this function will be called simultaneously by 2 different threads
+        # (SyncThreads) when 'heartbeat' and 'report' messages arrived at the same time.


line too long (88 > 79 characters)

houndci-bot · 2018-01-13T23:21:43Z

homeassistant/components/xiaomi_aqara.py

-        is_voltage = self.parse_voltage(data)
-        if is_data or is_voltage:
-            self.schedule_update_ha_state()
+        # There is a chance this function will be called simultaneously by 2 different threads


line too long (94 > 79 characters)

houndci-bot · 2018-01-13T23:21:43Z

homeassistant/components/xiaomi_aqara.py

+        if self._remove_unavailability_tracker:
+            self._remove_unavailability_tracker()
+        self._remove_unavailability_tracker = async_track_point_in_utc_time(
+            self._hass, self._set_unavailable, utcnow() + TIME_TILL_UNAVAILABLE)


line too long (80 > 79 characters)

houndci-bot · 2018-01-13T23:21:43Z

homeassistant/components/xiaomi_aqara.py

 from homeassistant.const import (ATTR_BATTERY_LEVEL, EVENT_HOMEASSISTANT_STOP,
-                                 CONF_MAC, CONF_HOST, CONF_PORT)
+                                 CONF_MAC, CONF_HOST, CONF_PORT, STATE_UNAVAILABLE)


line too long (83 > 79 characters)

houndci-bot · 2018-01-13T23:21:43Z

homeassistant/components/switch/xiaomi_aqara.py

            elif model == '86plug':
                devices.append(XiaomiGenericSwitch(device, 'Wall Plug',
-                                                   'status', True, gateway))
+                                                   'status', True, hass, gateway))


line too long (82 > 79 characters)

houndci-bot · 2018-01-13T23:33:50Z

homeassistant/components/xiaomi_aqara.py

+        self._remove_unavailability_tracker = None
+        self._state = STATE_UNAVAILABLE
+        self.schedule_update_ha_state()
+


blank line contains whitespace

Danielhiversen · 2018-01-22T08:08:19Z

Great.
What is wrong with Travis?

PaulAnnekov · 2018-01-22T20:47:29Z

@Danielhiversen it's ready to be merged. Merge my PR in PyXiaomiGateway, I will update version here.
P.S. I don't understand how could I add this old submodule in my commit f51ff3d#diff-d3eb3a918c38d80bb0a48ed78d8488ef

PaulAnnekov · 2018-01-22T22:29:29Z

@Danielhiversen you can merge it.

balloob · 2018-01-23T07:18:35Z

homeassistant/components/xiaomi_aqara.py

+        self._is_available = False
+        self.async_schedule_update_ha_state()
+
+    def _track_unavailable(self):


Please add the callback annotation and add async in the naem: _async_track_unavailable.

Done.
I thought these annotations and prefixes are required only when methods are called by internal HA functions, because they check them (at least annotations).

balloob · 2018-01-23T07:18:56Z

homeassistant/components/xiaomi_aqara.py

@@ -225,13 +247,33 @@ def device_state_attributes(self):
        """Return the state attributes."""
        return self._device_state_attributes

-    def push_data(self, data):
+    @callback
+    def _set_unavailable(self, now):


Please rename this to _async_set_unavailable

balloob

Two small comments. Can be merged after they have been addressed 👍 Awesome work 🐬

Danielhiversen · 2018-01-23T09:22:35Z

👍

MartinHjelmare · 2018-01-23T11:45:47Z

homeassistant/components/xiaomi_aqara.py

        self.parse_voltage(device['data'])

+    def _add_push_data_job(self, *args):
+        self.hass.async_add_job(self.push_data, *args)


Following the source, it looks like this callback will be called in a thread, so here we should use self.hass.add_job, and not the async version of the same method.

Yes, it will be called in a thread. I will fix it in 5 hours.

PaulAnnekov added 19 commits December 4, 2017 01:11

added unavailability tracker, updated sensor component

98b933c

change hass argument position according to position in binary_sensor

a4c364d

added hass argument to binary_sensor, updated is_on(), it can be UNAV…

f1e4bcc

…AILABLE now

updated switch component to support unavailability feature

e88b9e9

updated light component to support unavailability feature

2be53f9

updated cover component to support unavailability feature

4bac87d

set _hass property

87434f5

added unavailability tracker, updated sensor component

d13dc37

change hass argument position according to position in binary_sensor

07a0213

added hass argument to binary_sensor, updated is_on(), it can be UNAV…

ac6150a

…AILABLE now

updated switch component to support unavailability feature

661ceb9

updated light component to support unavailability feature

078522b

updated cover component to support unavailability feature

96a1503

set _hass property

16df8df

sync with upstream/dev

5c3341f

fixed error with wrong arguments number during callback call

3385020

reset unavailability state on new message received from device

e687288

use locks to fix race condition during managing _state property

ada08f4

overriden state() method for some components to check for STATE_UNAVA…

b92197e

…ILABLE and return it instead e.g. STATE_OFF

PaulAnnekov requested a review from Danielhiversen as a code owner January 13, 2018 23:21

homeassistant added integration: xiaomi_aqara platform: binary_sensor.xiaomi_aqara cla-signed labels Jan 13, 2018

houndci-bot reviewed Jan 13, 2018

View reviewed changes

fixed linter

d43a512

houndci-bot reviewed Jan 13, 2018

View reviewed changes

Danielhiversen closed this Jan 22, 2018

Danielhiversen reopened this Jan 22, 2018

removed accidentally added home-assistant-polymer

a7fab46

homeassistant added the cla-signed label Jan 22, 2018

bump PyXiaomiGateway version to 0.8.0

c7f03bd

homeassistant added the cla-signed label Jan 22, 2018

PaulAnnekov closed this Jan 22, 2018

PaulAnnekov reopened this Jan 22, 2018

bump PyXiaomiGateway to 0.8.0

d821db3

PaulAnnekov requested a review from andrey-git as a code owner January 22, 2018 21:57

homeassistant added the cla-signed label Jan 22, 2018

balloob reviewed Jan 23, 2018

View reviewed changes

balloob requested changes Jan 23, 2018

View reviewed changes

updated methods names and annotations

7d778ff

homeassistant added the cla-signed label Jan 23, 2018

Danielhiversen merged commit 3417c6a into home-assistant:dev Jan 23, 2018

MartinHjelmare reviewed Jan 23, 2018

View reviewed changes

PaulAnnekov mentioned this pull request Jan 23, 2018

Fix xiaomi_aqara.py async function called in a thread #11882

Merged

balloob mentioned this pull request Jan 26, 2018

0.62 #11921

Merged

home-assistant locked and limited conversation to collaborators May 29, 2018

ghost removed platform: binary_sensor.xiaomi_aqara labels Mar 21, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Checking Xiaomi Aqara devices unavailability states #11631

Checking Xiaomi Aqara devices unavailability states #11631

PaulAnnekov commented Jan 13, 2018

houndci-bot Jan 13, 2018

houndci-bot Jan 13, 2018

houndci-bot Jan 13, 2018

houndci-bot Jan 13, 2018

houndci-bot Jan 13, 2018

houndci-bot Jan 13, 2018

Danielhiversen commented Jan 22, 2018

PaulAnnekov commented Jan 22, 2018

PaulAnnekov commented Jan 22, 2018

balloob Jan 23, 2018

PaulAnnekov Jan 23, 2018

balloob Jan 23, 2018

PaulAnnekov Jan 23, 2018

balloob left a comment

Danielhiversen commented Jan 23, 2018

MartinHjelmare Jan 23, 2018

PaulAnnekov Jan 23, 2018

MartinHjelmare Jan 23, 2018

Checking Xiaomi Aqara devices unavailability states #11631

Checking Xiaomi Aqara devices unavailability states #11631

Conversation

PaulAnnekov commented Jan 13, 2018

Description:

Why is it needed?

How is it working

How to get notification when device becomes unavailable?

Average heartbeat time periods

Checklist:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Danielhiversen commented Jan 22, 2018

PaulAnnekov commented Jan 22, 2018

PaulAnnekov commented Jan 22, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

balloob left a comment

Choose a reason for hiding this comment

Danielhiversen commented Jan 23, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Average `heartbeat` time periods