zigbee2mqtt/bridge/state 'offline' after Mosquitto MQTT broker restart

[coldfire84]

What happened

I recently noticed all of my zigbee2mqtt devices became unavailable in Home Assistant, and traced the trigger back to a Mosquitto MQTT broker restart.

Restarting HA did not fix the problem, but restarting zigbee2mqtt did resolve the issue. On further investigation, I found that after restarting the broker zigbee2mqtt/bridge/state remains offline.

Looking through the logs I can see that zigbee2mqtt reconnects, as you would expect, but state is not updated on reconnect:

Zigbee2MQTT:error 2020-11-02 18:28:30: Not connected to MQTT server!
Zigbee2MQTT:info  2020-11-02 18:28:35: Connected to MQTT server

zigbee2mqtt itself is working, I can control devices via the zigbee2mqtt web GUI/ MQTT commands, just Home Assistant thinks all of the zigbee2mqtt devices are offline due to zigbee2mqtt/bridge/state being offline

What did you expect to happen

zigbee2mqtt should (imo!) reconnect when the MQTT broker is available and publish an zigbee2mqtt/bridge/state online message.

On startup/ initial connect I can see that zigbee2mqtt does publish to zigbee2mqtt/bridge/state:

Zigbee2MQTT:info  2020-11-02 20:02:42: Connecting to MQTT server at mqtt://hostname:1883
Zigbee2MQTT:debug 2020-11-02 20:02:42: Using MQTT keepalive: 30
Zigbee2MQTT:info  2020-11-02 20:02:42: Connected to MQTT server
Zigbee2MQTT:info  2020-11-02 20:02:42: MQTT publish: topic 'zigbee2mqtt/bridge/state', payload 'online'

However there is no such message publishing on a reconnect.

How to reproduce it (minimal and precise)

Restart Mosquitto MQTT broker without restarting zigbee2mqtt.

Cleaned config (removed device-specific/ network key) - note that MQTT version 5 in use for device-specific retention - hopefully I haven't missed anything obvious in my configuration, apologies if I have 🤞

mqtt:
  base_topic: zigbee2mqtt
  server: 'mqtt://hostname:1883'
  user: <username>
  password: <password>
  keepalive: 30
  version: 5
serial:
  port: /dev/ttyUSB0
frontend:
  port: 8081
experimental:
  new_api: true
advanced:
  log_level: debug
  homeassistant_discovery_topic: 'homeassistant'
  homeassistant_status_topic: 'homeassistant/status'
  rtscts: false
  report: true
  last_seen: ISO_8601

Debug info

Zigbee2MQTT version: 1.16.0 (I have also tested/ replicated this issue with 1.15.0, both containerised)
Adapter hardware: zzh! / CC26X2R1
Adapter firmware version: CC26X2R1_20201017.hex (from dev branch)
Mosquitto MQTT broker version: 1.6.12

[HolyEffendi]

It's the same for me. Can't connect to MQTT anymore. Reverted back to 1.15

[coldfire84]

I'm able to connect to my MQTT broker with zigbee2mqtt 1.16, this issue just relates to zigbee2mqtt/bridge/state not being updated to 'online' after a reconnect event (which also affects 1.15 from my testing).

[HolyEffendi]

Full snapshot recovery in my case

[zfrank2601]

I had the same problem, but it was fixed after I set username/password in configuration.yaml

[joad82]

Experiencing the same issue when having the new_api: true in place
When removing that the zigbee2mqtt service is starting up fine without any issues

[coldfire84]

Experiencing the same issue when having the new_api: true in place
When removing that the zigbee2mqtt service is starting up fine without any issues

Want to be sure we're talking about the same issue here.

Since upgrade to 1.16 (and with version 1.15) I have always been able to start zigbee2mqtt / connect to MQTT broker, with username and password. My issue is that if I restart the MQTT broker the zigbee2mqtt bridge status remains 'offline' in the retained messages on the broker despite zigbee2mqtt reconnecting to the broker, this is 'resolved' by restarting zigbee2mqtt.

[docBliny]

I also had the issue of not being able to connect to MQTT after updating to 1.16. I added my MQTT user's password to the config (username was already set) which fixed it. Thanks @zfrank2601 !

[joad82]

Experiencing the same issue when having the new_api: true in place
When removing that the zigbee2mqtt service is starting up fine without any issues

Want to be sure we're talking about the same issue here.

Since upgrade to 1.16 (and with version 1.15) I have always been able to start zigbee2mqtt / connect to MQTT broker, with username and password. My issue is that if I restart the MQTT broker the zigbee2mqtt bridge status remains 'offline' in the retained messages on the broker despite zigbee2mqtt reconnecting to the broker, this is 'resolved' by restarting zigbee2mqtt.

Alright, then we got different issues. Please ignore my comment

[VesleSlotten]

More or less same issue for me. If zigbee2mqtt lose its connection to MQTT server zigbee2mqtt/bridge/state becomes "offline" (probably the "will message"). When zigbe2mqtt reconnects the will message is not updated.

It's possible to set zigbee2mqtt/bridge/state to "online" manually (I use MQTT.fx to do this), and all devices becomes available in HA again.

[jacekpaszkowski]

I have the same problems as he described by @coldfire84 in this issue.

[jd766]

Have the same issue, happens by itself:

Zigbee2MQTT:info  2020-11-07 05:12:32: MQTT publish: topic 'zigbee2mqtt/computer_damiano', payload '{"linkquality":102,"power":9,"state":"ON"}'
Zigbee2MQTT:error 2020-11-07 05:12:38: Not connected to MQTT server!
Zigbee2MQTT:error 2020-11-07 05:12:38: Cannot send message: topic: 'zigbee2mqtt/computer_damiano', payload: '{"linkquality":102,"power":7,"state":"ON"}
Zigbee2MQTT:error 2020-11-07 05:12:42: Not connected to MQTT server!
Zigbee2MQTT:info  2020-11-07 05:12:43: Connected to MQTT server
Zigbee2MQTT:info  2020-11-07 05:12:44: MQTT publish: topic 'zigbee2mqtt/Wattmetro_Casa', payload '{"linkquality":87,"power":92,"power_alarm_active":false,"power_alarm_active_value":0,"power_alarm_enabled":0,"power_alarm_wh_threshold":0}'

At the same time, the Mosquitto HA addon shows this:

1604722245: Client mqttjs_67a4784b has exceeded timeout, disconnecting.
1604722353: New connection from 172.30.33.2 on port 1883.
[INFO] found mosqui on Home Assistant
1604722363: New client connected from 172.30.33.2 as mqttjs_67a4784b (p2, c1, k60, u'mosqui').
1604722418: Saving in-memory database to /data/mosquitto.db.

1604722245 converts to 05:10:45 CET. At 5:10:47, "sensor.bridge_state" turned to "offline" in the HA logbook

Connecting to the MQTT server showed zigbee2mqtt/bridge/state as offline, but the topics for the various devices were still being updated. I manually changed the topic to online and everything started working again.

[cyberdie]

Same problem here.
zigbee2mqtt bridge state goes offline and never comes back even when zigbee2mqtt and the broker (mosquitto) are working fine.
I've had user/pass in my config since the beginning of the setup so this is not a solution for me...

[Koenkk]

Should be fixed now.

Changes will be available in the latest dev branch in a few hours (https://www.zigbee2mqtt.io/how_tos/how-to-switch-to-dev-branch.html)

[juslex]

Dear @Koenkk

They are on the updated dev branch last Sunday after their fix goes live. However, today the problem has reappeared. When I restart the Home Assistant, the entities are unavailable, to resolve, I need to stop zigbee2mqtt and start again. I'm using the Bare-Metal installation.

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days

[mjozefcz]

I had the same issue and I wasn't able to fix it the same way its described here.

I fixed that by my own with (zigbee2mqttassistant installed via hassio):

1. Check where the configuration is bound.
   docker ps | grep zigbee2mqttassistant | awk {'print $NF'} | xargs docker inspect | grep data:rw

2. Manually modify the options.json file:
   /usr/share/hassio/addons/data/806b11b5_zigbee2mqttassistant-dev/options.json

3. Set chattr +i on file to prevent wiping the file on container start.
   chattr +i /usr/share/hassio/addons/data/806b11b5_zigbee2mqttassistant-dev/options.json

[bachoo786]

hey @coldfire84 did you manage to resolve this issue? I experience the same and I am running zigbee2mqtt version 1.21.1

[geeano]

same happened today with my upgrade to latest version of zigbee2mqtt

[fredronnv]

I'm experiencing this issue which is easily reproducible. Restart the mqtt broker and the bridge/state topic will change to offline. When zigbee2mqtt reconnects, the offline message is published shortly after the online message, causing integrations to fail.

$ mosquitto_sub -h <snip> -t 'zigbee/bridge/state' | ts
Apr 24 09:51:42 online
Apr 24 09:51:54 offline <- restart of mosquitto broker
Apr 24 09:51:55 offline
Apr 24 09:51:55 online
Apr 24 09:52:00 offline

Currently using the latest tag, koenkk/zigbee2mqtt latest 1e1571318897

[HeedfulCrayon]

I am also experiencing this issue. It is quite annoying

[xhemp]

Same here, just noticed this testing a HA configuration on nginx for MQTT.

[jpoeppelman1]

I have the same issue as well.

[fredronnv]

In case anyone needs a workaround, I run this every minute

                #!/usr/bin/bash
                HOST="<insert mqtt host>"
                TOPIC="zigbee/bridge/state"
                COMMAND="mosquitto_sub"
                COMMAND="$COMMAND -h $HOST -t zigbee/bridge/state -C 1 -W 1"
                STATE=$($COMMAND)
                if [ "$STATE" = "offline" ]; then
                    echo "Restart zigbee2mqtt due to bridge marked offline"
                    docker restart zigbee2mqtt
                fi

[Orochica]

I have the same issue

[fredskis]

Happened to me a couple of times now and is frustrating as I have one VM running Home Assistant OS and Mosquitto and Zigbee2Mqtt on another machine. Whenever the VM reboots all my Zigbee devices appear unavailable in Home Assistant until I restart Zigbee2Mqtt.
Seen a few workarounds and I'm probably going to set up a Home Assistant automation to send a blank payload to zigbee2mqtt/bridge/request/restart on Home Assistant start but ideally would like to not need to do this.

[jpoeppelman1]

I think this is fixed in the last release, mine is working now

[fredskis]

Ah yes I see it here: #9629
Thanks, I'll update now!

[hmoffatt]

I am still seeing this. I have 1.27.0.

[fredskis]

I was on 1.27.2 and still seeing it. Didn't seem to be as frequent as before the "fix" was implemented.
I upgraded to1.28.2 a couple of days ago and will try and remember to report back if I see it again.

[hmoffatt]

Thanks @fredskis , I just upgraded to 1.28.2 and a quick restart of mosquitto worked ok.

[hmoffatt]

Now after broker restart I see that it is connected but the z2m version and co-ordinator version fields are Unknown.

[bojanrajkovic]

I'm still seeing this on 1.28.2 — I set up a push notification from Home Assistant when a Z2M entity goes "Unavailable," and it happens every few hours.

Usually, it recovers after a few minutes, but sometimes it won't recover — for example, when I updated my Home Assistant OS, it never recovered, and I had to restart Z2M to get it to come back.

I have a couple of very noisy Tuya Air Quality Sensor devices, and I'm wondering if those are causing a memory leak somewhere or a long garbage collection period, and that's leading to a timeout.