Aqara binary_sensor entities getting renamed to "Door", in "Unavailable" state from Zigbee2MQTT after restarting HA

[stevehollaar]

The problem

After the recent update, my Aqara door sensors are not working in Home Assistant, though I see them working just fine in Zigbee2MQTT.

The binary_sensor entities have been all been renamed to "Door" on the Home Assistant side, despite retaining their correct descriptive names in Zigbee2MQTT. When I update their name in Zigbee2MQTT to re-sync their names, the entities are fixed, but only until the next time I restart Home Assistant, at which point they all become "Door" again.

What version of Home Assistant Core has the issue?

core-2023.9.0

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

MQTT

Link to integration documentation on our website

https://www.home-assistant.io/integrations/mqtt/

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

I saw the following in `ha core logs` after a restart of home assistant:


2023-09-07 22:49:17.755 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{ value_json.battery }
2023-09-07 22:49:17.758 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value json' is undefined when rendering '{{ value json.contact }}
2023-09-07 22:49:17.758 ERROR (MainThread) [homeassistant.helpers.template] Error parsing value: 'value_json' is undefined (value: , template: {{ value_json.contact }})
2023-09-07 22:49:17.765 ERROR (MainThread) [homeassistant.til. logging] Exception in state_message_received when handling msg on 'zigbee2mqtt/0×00158d000894c618':
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/mqtt/debug_info.py", line 44, in wrapper 
    msg_callback(msg)
  File "/usr/src/homeassistant/homeassistant/components/mqtt/binary_sensor.py", line 218, in state_message_received
    self._config [CONF_NAME],
KeyError: 'name'

2023-09-07 22:49:17.765 ERROR (MainThread) [homeassistant. helpers.template] Template variable error: 'value_json' is undefined when rendering '{{ value_json. device_temperature }}'
2023-09-07 22:49:17.766 ERROR (MainThread) [homeassistant. helpers. template] Template variable error: 'value_json' is undefined when rendering '{{ value_json. power_outage_count }}
2023-09-07 22:49:17.766 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{ value_json. state }}'
2023-09-07 22:49:17.767 ERROR (MainThread) [homeassistant.helpers.template] Error parsing value: 'value_json' is undefined (value: , template: {{ value_json. state }})
2023-09-07 22:49:17.767 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{ value_json. state }}'
2023-09-07 22:49:17.767 ERROR (MainThread) [homeassistant. helpers. template] Error parsing value: 'value_json' is undefined (value: , template: {{ value_json. state 77)
2023-09-07 22:49:17.767 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{ value_json.state }}'
2023-09-07 22:49:17.768 ERROR (MainThread) [homeassistant.helpers.template] Error parsing value: 'value_json' is undefined (value: , template: {{ value_json.state }})
2023-09-07 22:49:17.768 ERROR (MainThread) [homeassistant. helpers.template] Template variable error: 'value_json' is undefined when rendering '{{ value_json. state }}'
2023-09-07 22:49:17.769 ERROR (MainThread) [homeassistant.helpers. template] Error parsing value: 'value json' is undefined (value: , template: {{ value json. state }})

Additional information

No response

[home-assistant]

Hey there @emontnemery, @jbouwh, mind taking a look at this issue as it has been labeled with an integration (mqtt) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of mqtt can trigger bot actions by commenting:

• @home-assistant close Closes the issue.
• @home-assistant rename Awesome new title Renames the issue.
• @home-assistant reopen Reopen the issue.
• @home-assistant unassign mqtt Removes the current integration label and assignees on the issue, add the integration domain after the command.

_{^{(message by CodeOwnersMention)}}

---

mqtt documentation
mqtt source
_{^{(message by IssueLinks)}}

[jbouwh]

This looks more like an issue with Zigbee2Mqtt. If you believe it must be an issue with MQTT then the following is needed:

• A config payload found under topic homeassistant/binary_sensor/.
• The last state payload published.
  With that we can simulate what happens when HA is restarting or MQTT is reloading.

[jbouwh]

There is one issue though with the HA MQTT integration that needs to be solved. It seems that the name field is not set, which is a valid option. It will cause that the name will be set to that derived from the device_class. See the MQTT documentation for more info.
The KeyError in the logs should be solved for cases where no name is set.
It might be that this will solve your issue too.

[jbouwh]

Let's see if #99936 fixes the issue.

[jbouwh]

@stevehollaar van you confirm the fix did solve your issue?

[stevehollaar]

@stevehollaar van you confirm the fix did solve your issue?

I don't see that particular error in the logs, but my main issue of sensor entities getting renamed to "Door" and being unavailable after restarting is not fixed

[jbouwh]

To be able to reproduce that, more information is needed.

[stevehollaar]

I think it's related to this Z2M issue: Koenkk/zigbee2mqtt#18862 (comment)
I notice if I restart the Zigbee2MQTT add-on, the entities are available again. My issues come back if I then restart home assistant core without a full reboot of add-ons.

[jbouwh]

Right, let me know if this needs attention for HA core. Do you agree on closing this issue?

[stevehollaar]

I'm unsure if this is an issue with HA, Z2M or Mosquitto. Any pointers on how to determine that would be appreciated

[jbouwh]

Step 1 is find the configuration payload sent for the failing entity found under topic homeassistant/binary_sensor/some-id/config
You can use MQTT explorer or the tools found at the MQTT config entry settings.
Step 2. Is the payload that is send to the state topic and the debug logging when that happens.

[stevehollaar]

I think I may have something. I'll share some state from MQTT Explorer:

Timeline of actions:
~9:08pm: I restart Zigbee2MQTT addon: door sensor becomes available and working normally
~9:15pm: I restart Home Assistant core: door sensor becomes unavailable
~9:21pm: I restart Zigbee2MQTT addon: door sensor becomes available again

I can see under the zibee2mqtt/pantry_door_sensor topic 3 messages, corresponding to each action above:

however
under homeassistant/binary_sensor/<id>/contact/config I only see messages for the 2 Z2M restart actions:

Is that discrepancy of 2 vs 3 messages on the different topics the source of the entity going unavailable?

[jbouwh]

The config message does not change, and there should be one message per entity. Can you copy past this and a state message of the sensor. This message will change when the sensor changes. I'd like a copy of this message too so I can reproduce the issue

[stevehollaar]

Here's the value of homeassistant/binary_sensor/0x00158d000894c618/contact/config:

{"availability":[{"topic":"zigbee2mqtt/bridge/state","value_template":"{{ value_json.state }}"},{"topic":"zigbee2mqtt/pantry_door_sensor/availability","value_template":"{{ value_json.state }}"}],"availability_mode":"all","device":{"identifiers":["zigbee2mqtt_0x00158d000894c618"],"manufacturer":"Xiaomi","model":"Aqara door & window contact sensor (MCCGQ11LM)","name":"pantry_door_sensor","sw_version":"3000-0001"},"device_class":"door","object_id":"pantry_door_sensor_contact","origin":{"name":"Zigbee2MQTT","sw":"1.33.0","url":"https://www.zigbee2mqtt.io"},"payload_off":true,"payload_on":false,"state_topic":"zigbee2mqtt/pantry_door_sensor","unique_id":"0x00158d000894c618_contact_zigbee2mqtt","value_template":"{{ value_json.contact }}"}

and the value of zigbee2mqtt/pantry_door_sensor:

{"battery":90,"contact":true,"device_temperature":30,"linkquality":47,"power_outage_count":35,"voltage":2985}

Please let me know if I can provide anything else to debug this 🙏🏻

[jbouwh]

Thnx, that is all that I need for now.

[jbouwh]

The expected friendly name to be set should be:
pantry_door_sensor Door (assuming English as language set)
The name attribute on the entity is not set, so it inherits the name from the device_class.
If the name option would have been set to null, the friendly name would become pantry_door_sensor which is set as device name.

The only bug I discovered is, that when the config is updated and name is removed it does not remove the name attribute until HA is restarted.

So the Door entity should always be set based on the config received from Zigbee2MQTT available or not.

[jbouwh]

What I do not get is where this renaming takes place. May be Z2M has changed the config and entities are now appearing different. Is is a consequence of MQTT alligning with the HA naming convention. As I do not see any issues with MQTT I suggest closing this issue here.