MQTT json attributes #11439

tim-devel · 2018-01-03T21:06:27Z

Description:

Based on #10753. Allow MQTT sensor to process json values as attributes.

Related issue (if applicable): fixes

Pull request in home-assistant.github.io with documentation (if applicable): home-assistant/home-assistant.io#4334

Example entry for `configuration.yaml` (if applicable):

Checklist:

If user exposed functionality or configuration variables are added/changed:

Documentation added/updated in home-assistant.github.io

If the code communicates with devices, web services, or third-party tools:

Local tests with tox run successfully. Your PR cannot be merged unless tests pass
New dependencies have been added to the REQUIREMENTS variable (example).
New dependencies are only imported inside functions that use them (example).
New dependencies have been added to requirements_all.txt by running script/gen_requirements_all.py.
New files were added to .coveragerc.

If the code does not interact with devices:

Local tests with tox run successfully. Your PR cannot be merged unless tests pass
Tests have been added to verify that the new code works.

houndci-bot · 2018-01-03T21:06:32Z

homeassistant/components/sensor/mqtt.py

@@ -68,7 +72,7 @@ class MqttSensor(MqttAvailability, Entity):

    def __init__(self, name, state_topic, qos, unit_of_measurement,
                 force_update, expire_after, value_template,
-                 availability_topic, payload_available, payload_not_available):
+                 json_attributes, availability_topic, payload_available, payload_not_available):


line too long (96 > 79 characters)

tim-devel · 2018-01-03T21:08:13Z

I will write test and docs later this week

tinloaf · 2018-01-03T22:20:56Z

homeassistant/components/sensor/mqtt.py

@@ -104,6 +111,20 @@ def message_received(topic, payload, qos):
                self._expiration_trigger = async_track_point_in_utc_time(
                    self.hass, self.value_is_expired, expiration_at)

+            if self._json_attributes:


Nitpick: I personally strongly prefer if len(self._json_attributes) > 0. I'm always getting confused and assume things that are being used as boolean expression to be booleans.

Done, will upload in next commit

No, the pythonic way is to do if self._json_attributes to check if the container is non empty. It's also the preferred way for home assistant.

tinloaf · 2018-01-03T22:22:37Z

homeassistant/components/sensor/mqtt.py

@@ -104,6 +111,20 @@ def message_received(topic, payload, qos):
                self._expiration_trigger = async_track_point_in_utc_time(
                    self.hass, self.value_is_expired, expiration_at)

+            if self._json_attributes:
+                self._attributes = {}


Performance nitpick: Put this in the except clause to avoid setting the attribute twice.

Sorry but not sure how to do this. This feature will allow multiple attributes that could all be updated at different times, i.e. MQTT message 1 updates attribute 1 only, MQTT message 2 updates attribute 2 only, MQTT message 3 updates attribute 1 and 3 only

Oh yes of course. Makes sense.

tinloaf · 2018-01-03T22:28:33Z

homeassistant/components/sensor/mqtt.py

+                try:
+                    json_dict = json.loads(payload)
+                    if isinstance(json_dict, dict):
+                        attrs = {k: json_dict[k] for k in self._json_attributes


And while we're at it, another performance nitpick: make self._json_attributes a set and then use:
attrs = {k : json_dict[k] for k in self._json_attributes & json_dict.keys()}

Done, will upload in next commit

houndci-bot · 2018-01-04T18:33:04Z

homeassistant/components/sensor/mqtt.py

+                    json_dict = json.loads(payload)
+                    if isinstance(json_dict, dict):
+                        attrs = {k : json_dict[k] for k in self._json_attributes & json_dict.keys()}
+#                        attrs = {k: json_dict[k] for k in self._json_attributes


line too long (80 > 79 characters)

houndci-bot · 2018-01-04T18:33:05Z

homeassistant/components/sensor/mqtt.py

+                try:
+                    json_dict = json.loads(payload)
+                    if isinstance(json_dict, dict):
+                        attrs = {k : json_dict[k] for k in self._json_attributes & json_dict.keys()}


whitespace before ':'
line too long (100 > 79 characters)

houndci-bot · 2018-01-04T18:40:36Z

homeassistant/components/sensor/mqtt.py

+                try:
+                    json_dict = json.loads(payload)
+                    if isinstance(json_dict, dict):
+                        attrs = {k: json_dict[k] for k in


local variable 'attrs' is assigned to but never used

houndci-bot · 2018-01-04T19:00:12Z

tests/components/sensor/test_mqtt.py

@@ -216,3 +216,23 @@ def test_custom_availability_payload(self):
    def _send_time_changed(self, now):
        """Send a time changed event."""
        self.hass.bus.fire(ha.EVENT_TIME_CHANGED, {ha.ATTR_NOW: now})
+
+    def test_setting_sensor_value_via_mqtt_json_message(self):


redefinition of unused 'test_setting_sensor_value_via_mqtt_json_message' from line 112

tim-devel · 2018-01-04T20:43:17Z

I think this is ready to go but it is failing tox. The failures do not look like they are linked to this component but please let me know if i am missing something.

tim-devel · 2018-01-05T13:02:35Z

Looks like tests are passing now. Is this ready for merge?

tinloaf · 2018-01-05T13:38:45Z

homeassistant/components/sensor/mqtt.py

@@ -104,6 +111,20 @@ def message_received(topic, payload, qos):
                self._expiration_trigger = async_track_point_in_utc_time(
                    self.hass, self.value_is_expired, expiration_at)

+            if len(self._json_attributes) > 0:
+                self._attributes = set({})


I think there was a misunderstanding: self._json_attributes (not self._attributes) should be made a set, so that the computation of self._json_attributes & json_dict.keys() is efficient (intersection of two ordered iterables should be linear…)

Making self._attributes a set instead a dict here will provoke errors when self._json_attributes & json_dict.keys() is empty, because then device_state_attributes() returns a set instead of a dict. Please add a test that tests this case, i.e., when the set of selected attributes and the set of keys in the JSON payload are disjunct.

Sorry, I am no python Pro. I have had another go and trying to make self._json_attributes a set.

I have also added another couple of tests based on REST json attribute commit

MartinHjelmare · 2018-01-06T03:28:34Z

homeassistant/components/sensor/mqtt.py

@@ -104,6 +111,20 @@ def message_received(topic, payload, qos):
                self._expiration_trigger = async_track_point_in_utc_time(
                    self.hass, self.value_is_expired, expiration_at)

+            if len(self._json_attributes) > 0:


Please revert to if self._json_attributes:.

houndci-bot · 2018-01-06T07:46:23Z

tests/components/sensor/test_mqtt.py

+                'state_topic': 'test-topic',
+                'unit_of_measurement': 'fav unit',
+                'value_template': '{{ value_json.val }}'
+                'json_attributes': 'val'


SyntaxError: invalid syntax

houndci-bot · 2018-01-06T07:46:23Z

tests/components/sensor/test_mqtt.py

+                         self.force_update)
+
+
+        mock_component(self.hass, 'mqtt')


too many blank lines (2)

houndci-bot · 2018-01-06T07:46:23Z

tests/components/sensor/test_mqtt.py

+                         state.attributes.get('val'))
+
+
+    @patch('homeassistant.components.sensor.mqtt._LOGGER')


too many blank lines (2)

houndci-bot · 2018-01-06T07:46:24Z

tests/components/sensor/test_mqtt.py

+                'state_topic': 'test-topic',
+                'unit_of_measurement': 'fav unit',
+                'value_template': '{{ value_json.val }}'
+                'json_attributes': 'val'


SyntaxError: invalid syntax

houndci-bot · 2018-01-06T07:46:24Z

tests/components/sensor/test_mqtt.py

+                         self.force_update)
+
+
+        mock_component(self.hass, 'mqtt')


too many blank lines (2)

houndci-bot · 2018-01-06T07:46:24Z

tests/components/sensor/test_mqtt.py

+                         state.attributes.get('val'))
+
+
+    @patch('homeassistant.components.sensor.mqtt._LOGGER')


too many blank lines (2)

tim-devel · 2018-01-06T08:16:01Z

Hmm, my tests don't work, will take another look later

tim-devel · 2018-01-07T22:27:44Z

I think this is ready now

j2mc · 2018-02-05T12:08:21Z

Is it possible to read nested JSON into attributes? With the RESTful sensor, if the json_attributes key's value contains JSON it creates an attribute for each nested key. See this post

MartinHjelmare · 2018-02-05T12:48:27Z

Please open an issue if you suspect a bug. If you need help please use our help channels:
https://home-assistant.io/help/#communication-channels

Merged PRs should not be used for support or bug reports. Thanks!

MQTT json attributes

7b352db

homeassistant added cla-signed platform: sensor.mqtt labels Jan 3, 2018

balloobbot added the platform label Jan 3, 2018

houndci-bot reviewed Jan 3, 2018

View reviewed changes

Fix lint

2169483

homeassistant added the cla-signed label Jan 3, 2018

tim-devel mentioned this pull request Jan 3, 2018

MQTT json sensor attributes home-assistant/home-assistant.io#4334

Merged

2 tasks

tinloaf reviewed Jan 3, 2018

View reviewed changes

Amends following comments

8a8aed1

houndci-bot reviewed Jan 4, 2018

View reviewed changes

Fix lint

e2155ff

houndci-bot reviewed Jan 4, 2018

View reviewed changes

timstanley1985 added 2 commits January 4, 2018 18:44

Fix lint

a83aa22

Add test

7f54913

houndci-bot reviewed Jan 4, 2018

View reviewed changes

Fix typo

d215501

tinloaf suggested changes Jan 5, 2018

View reviewed changes

MartinHjelmare reviewed Jan 6, 2018

View reviewed changes

timstanley1985 added 2 commits January 6, 2018 07:45

Amends following comments

0ea8942

New tests

5dd4dd8

houndci-bot reviewed Jan 6, 2018

View reviewed changes

Fix lint

4e282a6

Fix tests

b204a82

tinloaf approved these changes Jan 8, 2018

View reviewed changes

fabaff merged commit b1b0a25 into home-assistant:dev Jan 8, 2018

tim-devel deleted the mqtt_sensor_attributes branch January 8, 2018 21:18

balloob mentioned this pull request Jan 11, 2018

0.61 #11589

Merged

home-assistant locked and limited conversation to collaborators Feb 5, 2018

ghost added integration: mqtt and removed platform: sensor.mqtt labels Mar 21, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MQTT json attributes #11439

MQTT json attributes #11439

tim-devel commented Jan 3, 2018 •

edited

Loading

houndci-bot Jan 3, 2018

tim-devel commented Jan 3, 2018

tinloaf Jan 3, 2018

tim-devel Jan 4, 2018

MartinHjelmare Jan 6, 2018

tinloaf Jan 3, 2018

tim-devel Jan 4, 2018

tinloaf Jan 4, 2018

tinloaf Jan 3, 2018

tim-devel Jan 4, 2018

houndci-bot Jan 4, 2018

houndci-bot Jan 4, 2018

houndci-bot Jan 4, 2018

houndci-bot Jan 4, 2018

tim-devel commented Jan 4, 2018 •

edited

Loading

tim-devel commented Jan 5, 2018

tinloaf Jan 5, 2018

tim-devel Jan 6, 2018

MartinHjelmare Jan 6, 2018

houndci-bot Jan 6, 2018

houndci-bot Jan 6, 2018

houndci-bot Jan 6, 2018

houndci-bot Jan 6, 2018

houndci-bot Jan 6, 2018

houndci-bot Jan 6, 2018

tim-devel commented Jan 6, 2018

tim-devel commented Jan 7, 2018

j2mc commented Feb 5, 2018 •

edited

Loading

MartinHjelmare commented Feb 5, 2018

		state.attributes.get('val'))


		@patch('homeassistant.components.sensor.mqtt._LOGGER')

MQTT json attributes #11439

MQTT json attributes #11439

Conversation

tim-devel commented Jan 3, 2018 • edited Loading

Description:

Example entry for configuration.yaml (if applicable):

Checklist:

Choose a reason for hiding this comment

tim-devel commented Jan 3, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tim-devel commented Jan 4, 2018 • edited Loading

tim-devel commented Jan 5, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tim-devel commented Jan 6, 2018

tim-devel commented Jan 7, 2018

j2mc commented Feb 5, 2018 • edited Loading

MartinHjelmare commented Feb 5, 2018

tim-devel commented Jan 3, 2018 •

edited

Loading

Example entry for `configuration.yaml` (if applicable):

tim-devel commented Jan 4, 2018 •

edited

Loading

j2mc commented Feb 5, 2018 •

edited

Loading