deCONZ losing connectivity to end-devices

[ebaauw]

deCONZ v2.04.94 (and some earlier versions) seems to be losing connectivity to end devices. The end-devices (Hue motion sensors, Hue dimmer switches, IKEA dimmer) still appear on the deCONZ GUI, but they don't communicate with the ZigBee network anymore. They seem to have lost connectivity to their parent device and/or the parent device no longer forwards commands to the mesh network. I think this might be related to the mesh network discovery (the "IKEA mesh" problem, #195). Symptoms include:

• The Hue motion sensor blinks red when detecting motion. Attribute reports are no longer recevied by the gateway, state.lastupdated from all three resources is no longer updated. When reading the attributes from the deCONZ GUI, the node blinks red. The only remedy is hard resetting the motion sensor (holding the reset button for 10 seconds) and re-pairing it to the gateway. A simple reset (short press on the reset button doesn't help, nor does leaving/joining the network from the deCONZ GUI, nor restarting deCONZ, nor resetting the RaspBee.
• The Hue dimmer switch blinks red when a button is pressed. It's not just the notifications to the gateway that don't work; also the group commands don't work. After pressing all four buttons on the switch for a couple of seconds (simple reset), the switch works again.
• The IKEA dimmer switch seems to have gone mute. The only way to revive it is a hard reset (four times pressing the reset button) and re-pairing it to the gateway.
• The end devices seem to drop at random times, not related to pressing a button or detecting motion or restarting deCONZ. They seem to be taking turns.
• The end devices appear to be having multiple parents (multiple lines in the deCONZ GUI, and the deCONZ log reports multiple neighbours: "Node 0x001788xxxxxxxxxx is known by 3 neighbours". Typically these parents are (Philips Hue) lights, not the RaspBee itself.

[manup]

Which firmware version is your setup? There is a difference in mesh network handling between 0x26190500 and 0x261A0500.

Often a hard reset can be avoided by doing a power cycle, turn out battery, press buttons if available and put battery in again.

The Hue motion sensor (and dimmer?) expects a ZCL default response to a report and will blink red if this is not received, it then tries to reconnect which may cause searching a new parent node. I guess the multiple parents issue is the result of that. Timeout of end devices on parents is quite long.

Can you confirm that the ZigBee channel has not changed? The current firmware has a bug which happens for some users and results in channel changes (without raised network update id). This causes a lot of problems. Version 2.04.96 has code to prevent that and revert to the former channel when a unexpected channel change was detected. The next firmware will also prevent the issue from low level perspective.

[ebaauw]

Which firmware version is your setup?

I'm on 0x26190500. deCONZ doesn't prompt me to upgrade to 0x261A0500 - should I do that manually?

Often a hard reset can be avoided by doing a power cycle, turn out battery, press buttons if available and put battery in again.

I could try that next time. It does require a screwdriver for the Hue motion sensor, so hard-reset and re-pair might actually be less work.

The Hue motion sensor (and dimmer?) expects a ZCL default response to a report and will blink red if this is not received, it then tries to reconnect which may cause searching a new parent node. I guess the multiple parents issue is the result of that. Timeout of end devices on parents is quite long.

Yes, and I've seen the motions sensor finding a new parent (e.g. when old parent is powered off). Somehow, it doesn't find one in this case. I think the dimmer works the same: it finds a new parent when pressing/holding all four buttons.

Can you confirm that the ZigBee channel has not changed?

Yes. It's been on 25 for months.

Version 2.04.96

Yeah, I saw that one on the download server. Updating...

[marthoc]

I also noticed this issue with battery powered devices, particularly after a restart of the gateway (eg last night I updated to .96 and so restarted). After deCONZ came back up, my lights were perfectly controllable but my Hue Dimmer and IKEA motion sensor would not react (in the Phoscon web app at first these devices showed a grey icon, but after a short time the icon was not grey and the app reported that the motion sensor was ok, but it wouldn’t detect motion events). I left them like that overnight. This morning, the remote works and so does the motion sensor. So perhaps there is some delay after they rejoin the network?

[ebaauw]

So perhaps there is some delay after they rejoin the network?

ZigBee creates a mesh network, that continues to work without the gateway. When restarting deCONZ, the sensors don't rejoin the network, the RaspBee (or ConBee) does. To reset the network, you need to power down all its routers (lights).

When the gateway rejoins the network, it starts reading the neighbour tables of all routers (lights). The end devices (sensors) appear in the deCONZ GUI as soon as their parent is read. It's only when the sensor wakes up and sends a notification to the gateway, that config.reachable is set to true. If attribute reporting has be setup correctly, the notification for detecting motion actually triggers this. If attribute reporting has not been setup correctly, the notification won't arrive at the gateway. The gateway can only setup attribute reporting, when it knows the end device is awake, so another notification (e.g. reporting battery state, or a power cycle of the sensor) is needed.

[marthoc]

@ebauuw This is interesting and makes sense to me. What I’m seeing then is that though the remotes appear in deCONZ, they don’t start sending websocket events on the rest API for some time (currently testing deCONZ with @Kane610’s Home Assistant integration, which relies on the rest API).

[ebaauw]

Now on .97. Had another one of my Hue motion sensors dropping from the network. Removing and replacing batteries does not help. I did make sure to remove both of them.

[manup]

I wonder what happens here, can you please make a sniffer log when this happens again?

[snozzlebert]

Now on .97. Had another one of my Hue motion sensors dropping from the network. Removing and replacing batteries does not help. I did make sure to remove both of them.

Lost one of my two Hue motions sensors today. I'm on 0.97 and 0x261A0500

[esseki]

I'm on 0.97 and 0x261A0500

@snozzlebert : may i ask you were did you find this version of the firmware ? I'm running 0x26190500 with .97 and when i check i can not find it in /usr/share/deCONZ/firmware

FYI i'm using a Conbee and not a Raspbee

[marthoc]

Maybe unrelated, but I’m also unable to join a new Hue motion sensor to my network through deCONZ, I’m on .97

[manup]

Yes there is a bug in 97 preventing join the sensor. Fixed already, tommorow 98 will be released

[snozzlebert]

@snozzlebert : may i ask you were did you find this version of the firmware ? I'm running 0x26190500 with .97 and when i check i can not find it in /usr/share/deCONZ/firmware

#230 (comment)

[ebaauw]

I have another motion sensor going incommunicado. I physically moved the motion sensor, the RaspBee running deCONZ, and the ConBee running BitCatcher with 50cm of each other, yet I cannot seem to capture any frame.

When trying to read the sensor's attributes from the deCONZ GUI, I see no Read Attributes command, like I see when trying to read the attributes from another motion sensor. In the deCONZ output (--dbg-info=2 --dbg-aps=1) I see the request to read the attributes:

15:01:21:767 APS-DATA.request id: 40, addrmode: 0x02, addr: 0xe0ac, profile: 0x0104, cluster: 0x0400, ep: 0x02 queue: 4 len: 11
15:01:21:767 APS-DATA.request id: 41, addrmode: 0x02, addr: 0xe0ac, profile: 0x0104, cluster: 0x0400, ep: 0x02 queue: 5 len: 5
15:01:21:767 Delay APS request diff = 1

Not sure if the last line is related - I see more of these (which a different number after the =).

Somewhat later I see:

15:01:31:885 APS-DATA.confirm id: 40, status: 0xD0
15:01:32:394 APS-DATA.confirm id: 41, status: 0xD0

Searching for the sensor's NWK address (case independent) doesn't find anything. Searching for the MAC address yields:

14:58:10:188 Node 0x00178801020267E7 is known by 4 neighbors, last seen 10528 s
14:58:45:315 Node 0x00178801020267E7 is known by 4 neighbors, last seen 10564 s
14:59:20:356 Node 0x00178801020267E7 is known by 4 neighbors, last seen 10599 s
14:59:55:396 Node 0x00178801020267E7 is known by 4 neighbors, last seen 10634 s
15:00:30:540 Node 0x00178801020267E7 is known by 4 neighbors, last seen 10669 s
15:00:34:721 max transmit errors for node 0x00178801020267E7, last seen by neighbors 10673 s
15:00:35:241 max transmit errors for node 0x00178801020267E7, last seen by neighbors 10673 s
15:00:47:762 max transmit errors for node 0x00178801020267E7, last seen by neighbors 10686 s
15:00:48:282 max transmit errors for node 0x00178801020267E7, last seen by neighbors 10686 s
15:01:05:868 Node 0x00178801020267E7 is known by 4 neighbors, last seen 10704 s
15:01:30:054 max transmit errors for node 0x00178801020267E7, last seen by neighbors 10728 s
15:01:30:606 max transmit errors for node 0x00178801020267E7, last seen by neighbors 10729 s
15:01:31:886 max transmit errors for node 0x00178801020267E7, last seen by neighbors 10730 s
15:01:32:394 max transmit errors for node 0x00178801020267E7, last seen by neighbors 10731 s
15:01:40:908 Node 0x00178801020267E7 is known by 4 neighbors, last seen 10739 s
15:01:44:915 max transmit errors for node 0x00178801020267E7, last seen by neighbors 10743 s
15:01:45:474 max transmit errors for node 0x00178801020267E7, last seen by neighbors 10744 s
15:02:15:948 Node 0x00178801020267E7 is known by 4 neighbors, last seen 10774 s
15:02:51:068 Node 0x00178801020267E7 is known by 4 neighbors, last seen 10809 s
15:03:26:108 Node 0x00178801020267E7 is known by 4 neighbors, last seen 10844 s
15:04:01:259 Node 0x00178801020267E7 is known by 4 neighbors, last seen 10879 s

I cannot find any messages in the deCONZ log when removing/re-inserting the sensor's batteries.

Attached the sniffer log when re-inserting the batteries and waiving in front of the sensor (which then blinks red). Network key: 0x00d46c08d048986ca4fcbc9048fc082c.
sensor.dcf.zip

I cannot find any Report Attributes commands - as far as I can tell there's something wrong lower in the stack (maybe the sensor doesn't get a NWK address or something?).

[wvuyk]

Also here lost communication to a philips motion sensor. Restarting the raspbee, triggering the sensor, nothing seems to work. In the deCONZ interface it is present, but not connected to any router. Phoscon reports it as unreachable....

[wvuyk]

I have tried to make the motion sensor work again with .99, but no luck, removed the device in deCONZ, it would come back, even after a reset of the motion sensor. Deleted it with the REST API, the motion sensor would come back with the HA option block for occupancy and illuminance. the ZLL enpoints were not shown at all. A reset (f5) on the device could add Tempeterature if I was lucky.

There is no way I can bring this motion sensor back to life again... so I moved it back to an old Hue bridge. Worked immediatly like normal again.

Wim

[manup]

I think I know where the problem in deCONZ core application is, it could happen for all end devices, but is most dramatic for Philips Hue Dimmer switch and motion sensor. Looking forward to provide a fix during the next days.

[seydx]

Also have problems with all my hue dimmer swiches... after pressing one of the 4 buttons, it blinks red... need to press all 4 buttons to reset... and after couple of seconds/minutes, it blinks red again...

[rtenklooster]

Same problem with build 99 and my xiaomi sensors and Ikea bulbs.
They show up as unreachable but when they detect motion they shortly change from gray to solid black in the new webapp.

I did noticed this after upgrading from 97 to 99.

After disconnecting the Mi plug router all sensors are working fine.

[frahlg]

I seem to have the same issue. Just bought a Conbee and have both Trådfri and Hue lights i my network. I have two Philips motion sensors, one of them work but does not report temp (only 0 degrees). The other is only reporting lux, no precense or temp. I have tried resetting, re-pairing and removing battries more than once, still no luck. I have had problems with the Dimmer switches as well, but after a reset a they now seem to work.

[ebaauw]

I seem to have the same issue.

No it's not, see #355.

[wvuyk]

Today lost another motion sensor here on my raspbee running deCONZ 2.05.11. No information is communicated here. I tried resetting the motion sensor an rejoining the deCONZ. During the joining process i can see the blue dot blinking in deCONZ, but after that no more communication seems to be done anymore.

I moved the sensor to my Philips bridge and it works like a charm then again.
I wonder how one can remove a device from deCONZ completely, .e.g. forget it ever existed? As when i rejoin it is looks like it is already known? Is there a cleaning process other then the delete key for deCONZ?

[manup]

Is the RaspBee firmware 261E0500? You can see this in Phoscon App > Menu > Gateway.
To what parent is the sensor connected? The current state is that it can be tricky when the sensor selects a IKEA light as parent. If possible you can try to force the sensor to connect directly to RaspBee or Philips lights by power other lights off until the sensor is connected.

The sensor expects the gateway to be always on, if deCONZ is turned off for a few minutes the sensor will start to rejoin the network which may take a few minutes until everything works again.

How is the setup on the Philips bridge are there IKEA lights?

I wonder how one can remove a device from deCONZ completely, .e.g. forget it ever existed?

Delete it in the Phoscon App, this will delete all 3 sensor resources from the REST-API. The only thing still known is the position in deCONZ and endpoint information which shouldn't cause problems.

Also when adding the sensor to deCONZ please do it in the Phoscon App via the Philips Hue sensor wizard.

[wvuyk]

The raspbee firmware is 261E500 since a while here. The parent that it was connected was a GEN1 Hue color bulb. when it reconnected it was connected to the same light I believe.
There are two Ikea's in deCONZ and on the Philips bridge there are none.
Sensors are added here with the Phoscon app.

I just noticed that in deCONZ the sensor showed up again, although I had it deleted it in Phoscon and deCONZ. Could it be that the Philips bulb attached was in error forwarding the signals maybe? I have power cycled this one will try moving the sensor back again maybe?

I never checked back on the earlier motion sensor (earlier in this thread) that I could not rejoin anymore.

[manup]

The sensor is still known in the parent light for example hue lights keep end devices for a very long time and will report it in the neighbor table, hence it does pop up again but other than the zombie node shown in deCONZ it doesn't have an impact and can be ignored. When re-adding you better force the sensor to connect to RaspBee or the light directly by turning off IKEA lights power while adding (if it is location wise possible, prefer the RaspBee).

I hope the issues in mixed networks with end devices will be resolved in the next firmware versions of the related devices, there is currently some tests and exchange of information between vendors on that matter ...

[wvuyk]

Found out how to repair this. I switched off the Ikea's, like you said. But rejoining would just connect to the gen1 Philips bulb again, even while holding it close the RaspBee, and have no updates on the sensors anymore. So I did reset the motion sensor and I switched off the Philips bulb.
Then it connected to an Osram plug and would not deliver all the end points, even after waiting (pressing 0 many times and causing movement to the sensor) 15 minutes.
Then disconnected the Osram plug and again did a reset and rescan. Still, holding it close to the RaspBee, it connected to a GEN2 Philips GU10 color device. Immediatly the sensor was complete and it also updated sensors again.

Problem solved, will try tomorrow with the earlier sensor as well. But appearantly this motion sensor is quite stubborn when it comes to reporting correctly to different brands and - I think - older bulbs?

thanks!

[wvuyk]

After a few days the same sensor has stopped sendng updates again (the dot is not blinking in deCONZ).
I have checked, it was connected to a ikea bulb agan, so for some reason it changed tht connection again. Switched off the Ikea's again, did a leave/join of the network and reset the motion sensor. I am on 2.0.5.15 now.

When the sensor joins deCONZ, at the moment I see the double dot on the right appear, deCONZ restarts, without clear reason. After the restart, the sensor will not sens a visual signal to the dot - the double dot is single again, no end points available.
A new reset and a join will do exactly the same.

How can we proceed on this?

[manup]

Hi, deCONZ version 2.05.16 with firmware version 0x261f0500 should help to mitigate the problem. If it happens again please leave a short note here.

[wvuyk]

@manup Thanks. Have moved the sensors back to my raspbee and both are reporting correctly again. One sensor moved back without issues, the other (first mentioned in this thread) had some issues, like not updating motion but updating temperatures. Another reset and search on the raspbee solved the issue.

Will report back if issues occur.

[johncollaros]

So perhaps there is some delay after they rejoin the network?

ZigBee creates a mesh network, that continues to work without the gateway. When restarting deCONZ, the sensors don't rejoin the network, the RaspBee (or ConBee) does. To reset the network, you need to power down all its routers (lights).

When the gateway rejoins the network, it starts reading the neighbour tables of all routers (lights). The end devices (sensors) appear in the deCONZ GUI as soon as their parent is read. It's only when the sensor wakes up and sends a notification to the gateway, that config.reachable is set to true. If attribute reporting has be setup correctly, the notification for detecting motion actually triggers this. If attribute reporting has not been setup correctly, the notification won't arrive at the gateway. The gateway can only setup attribute reporting, when it knows the end device is awake, so another notification (e.g. reporting battery state, or a power cycle of the sensor) is needed.

This advice RE restarting all routers in the network did help for my unavailable IKEA sensors and a light to become available again.
Is there any network command in Zigbee which can do this reset in a soft way?
I am running HomeAssistant with deCONZ using a Conbee II.