How to support multi coordinator ?

[qlwz]

I have 50-60 devices, how can I support multiple coordinators

[Koenkk]

Multiple coordinator in a Zigbee network is not possible.

[dfsx1]

@Koenkk I was also thinking about setting this up after reading #3011.
Why is it not possible, what about multiple networks?

How I was thinking to implement this myself:

• insert 2x CC2531 adapters in Pi
• run 2 seperate zigbee2mqtt instance in docker
• configure both on different zigbee channels (so they don't conflict)

Then theoretically I should get 2 fully isolated zigbee networks right?
And with 4 USB ports in the Pi, I can scale up to 4 networks (later).

1x CC1352P-2 costs $50 (100+ devices)
1x CC2531 costs $3 (~30 devices)
4x CC2531 costs $12 (~120 devices)

However, I'm not yet familar with all topics over MQTT and if the HA integration would support multiple zigbee2mqtt instances communicating to it.
If not, I can maybe write a custom MQTT consumer/publisher to proxy / do some magic in between those instances?

[Koenkk]

Multiple networks is technically possible but I personally would never recommend it:

• The 2.4Ghz channel is already quite occupied in most regions, adding more separate Zigbee networks will use even more space in the 2.4Ghz range
• Your network will be less strong, less devices = smaller mesh
• Multiple zigbee2mqtt instances need to be maintained and updated
• As you already mentioned, it will make the MQTT topics and such more complex regarding HA integration, not sure how this will work, never done it.

If you want to create such a large network I would really recommend just getting the CC1352P-2 or CC26XR1, it will save you a lot of time and headache. If you are going to spend a lot of money on 100+ Zigbee devices, why would you even worry for +- 30 dollars extra for a decent coordinator?

[dfsx1]

Thanks for your reply :)
I can see my setup reach ~50 devices before end of the year.
I'm addicated now and AliExpress is cheap! 🥰

The 2.4Ghz channel is already quite occupied in most regions

My wifi is on 5Ghz, but I live in crowded area so I wanted Zigbee channel 25 for least interference on 2.4Ghz. But then some Ikea devices refused to join.
I should retry after recent OTA updates, but for now I went back to default channel 11.
In the docs I also came across the Konke devices which only work on channel 15, 20 and 25.

So I can see a reason to operate on multiple networks/channels.
For myself I want to run 11 and 25, based on my current device's needs.
I also want to make a high-available setup with Kubernetes and multiple Pi's (including battery-pack failover for my alarm).
With 2 channels on 2 Pi nodes, I need at least 4 adapters

why would you even worry for +- 30 dollars extra for a decent coordinator?

$12 (4x3) or $200 (4x50) is a big difference IMO.
I rather buy 20 more Zigbee devices for that $188.
And I already paid $15 for the CCDebugger, I want to use it more than once :)

not sure how this will work, never done it.

Especially because probably never done, I want to make this a personal goal :)
If it fails I only lose $3. I can't wait to start testing it.
Now first waiting 2 weeks for my (2nd) adapter from China 😞

[qlwz]

Multiple networks is technically possible but I personally would never recommend it:

• The 2.4Ghz channel is already quite occupied in most regions, adding more separate Zigbee networks will use even more space in the 2.4Ghz range
• Your network will be less strong, less devices = smaller mesh
• Multiple zigbee2mqtt instances need to be maintained and updated
• As you already mentioned, it will make the MQTT topics and such more complex regarding HA integration, not sure how this will work, never done it.

If you want to create such a large network I would really recommend just getting the CC1352P-2 or CC26XR1, it will save you a lot of time and headache. If you are going to spend a lot of money on 100+ Zigbee devices, why would you even worry for +- 30 dollars extra for a decent coordinator?

now , we have to launch multiple dockers，very troublesome to manage

[Koenkk]

The point is that you shouldn't use multiple coordinator, one CC1352P-2 or CC26X2R1 is enough for your whole network.

[tanus10]

Since we have serial2TCP option available now, we can think of a few scenarios that ZIGBEE2MQTT is handling more than a single meshed Zigbee network. (cf. adjacent buildings, different floors sharing the same wifi network, etc.)

serial:
    port: 'tcp://192.168.2.13:20108'

For me it's a 2 floors of reinforced concrete house, a single wifi network is done with ethernet cable connected APs, but no chance of Zigbee signals getting through - thick reinforced concrete walls and metal doors, wifi is not doing well, I had to install 4 wired APs - Not without using a few Zigbee routers set up along the walled/gated corridors and twisted stairs. (no power outlets along the stairs was the killing blow. btw, LAN cables are already installed in the wall)

I know CC26X2R might, yes might work, but I have several CC2531s and CC2530s lying around and wifi is already very reliable.

For now, I see that another instance of Docker container and cheap ESP01+cc2530 in each floor is the way to go. Before this, I had 2 Rpi's with cd2531+zigbee2mqtt running on each.

But still I wish zigbee2mqtt could handle multiple coordinators/networks.

[bojanpotocnik]

I also have a similar use case: we have a house and ~40 m away another building with garages, workshop and other farm related stuff, and ~40 m on the other side another one (stables, machinery, ...). I connected the buildings with ethernet cable and in every building the setup is at least:

• Mikrotik hAP ac³ (bought because of the USB port)
• Electrolama zig-a-zig-ah! dongle in the Mikrotik USB port
• Raw TCP access to serial port enabled

Therefore, I use 3 coordinators not because of device limit (zig-a-zig-ah! is CC2652R anyway) but as a similar reason to @tanus10 - I cannot install repeaters in between buildings, so the only other option would be to use powerful repeaters with directional antennas on the side of every building (point-to-point style), creating (physically) a very stretched network.

The problem I saw today when installing HA on a new machine is not managing multiple dockers, but if one would like to use Home Assistant Zigbee2mqtt Add-on, it is not even possible to run multiple instances of addon (each with different serial2TCP address).

Edit: maybe #861 would be a more fitting issue? 🤔

[timdonovanuk]

@Koenkk if switching between adapters with the same chips is supported without repairing then I honestly think it's time to revisit this. There is now could absolutely be a scenario where you have:

• z2m live
• z2m standby

The settings (and device list etc.) of the two instances are kept in sync via mqtt, but the z2m standby doesn't activate until it sees an MQTTT Last Will and Testament message 'from' the live instance (I believe these messages actually originate from the MQTT server when it notices an MQTT client is no longer reachable). This indicates z2m is down, which could trigger the standby instance to activate. Of course any z2m running in this model would need to check if there is already a z2m instance running.

[Koenkk]

@timdonovanuk although complex (and I don't see a good use case for it), this is already possible to setup.

[timdonovanuk]

@Koenkk we've all been on a long home automation journey the last few years, and there are many of us that have matured from a single Raspberry Pi setup to much more complex setups like Proxmox clusters, kubernetes swarms etc. with high availability and failover support. There is literally ONE massive achilles heel to even the best redundant setup and Zigbee is it. I appreciate its not for you to address the shortcomings of Zigbee Alliance and the protocol though.

I'll look into what is possible with the LWAT messages, thanks!

[tanus10]

Not strictly on the topic, but redundancy is not just a problem for a glorified big systems. I am hesitant to install a couple of zigbee bulbs at my old parents house (old age, need some timers on the bulb without new wiring, right now just using it for sensors), because they will be helpless if the bathroom bulb won't turn on. I am not saying redundancy will cover everything but, it may help, even for smaller, simple setup, and it will give you a little peace of mind extra.
It actually happened, one of zigbee coordinator (CC2530+ESP8266, remote serial-over-wifi connection) suddenly failed and z2m just produced tons of errors and anybody at my old man's house and (me at remote site) didn't noticed for days.

z2m have grown well, I think redundancy and scalability is a few of worthy considerations now on.

[timdonovanuk]

@tanus10 absolutely fair, and it's even easily affordable with the price of older Pi boards and cc2531 sticks.

[timdonovanuk]

FWIW, I just had my CC1352P-2 completely die. No board LED lights turn on, nothing recognises it etc. So I can now say 100% I have a good use-case for failover ;)

I was able to get up and running again with an old CC2531 stick (I had to delete coordinator_backup.json as the herdsman was trying to restore the backup and it was incompatible with a different stick I guess?).

I'm using ser2net now also. I can imagine several HA/failover scenarios using ser2net rather than relying on MQTT LWAT messages.

[timdonovanuk]

I've made some limited progress on this, but I'm pursuing a setup using two coordinator sticks across two Pi's.

Using the tool uhubctl you can turn off power to Pi's USB ports, in my case using the command:

sudo uhubctl -l 1-1 -p 2 -a 1

This should avoid any issues with two coordinators being powered on at the same time. I'm not sure where to take it from here but will report back :)

@Koenkk would this be better moved to a discussion? Or possible to re-open the issue? I'd love to hear others thoughts on it, or how they're thinking it might be implemented, even if it's not part of core implementation. Cheers!

[Koenkk]

@timdonovanuk might be better to create a discussion indeed.

For my information, what firmware were you using on the CC1352-P2?

[rjulius23]

@Koenkk the active/standby solution described by @timdonovanuk possible now ? How to handle the HW adress difference between the 2 coordinators (active and standby) ?

[Jpsy]

Have you seen this video?: https://youtu.be/KBEaVZyvSgY

It seems that HA can be easily tricked into installing Z2M multiple times and the incarnations can coexist without problems. The video answers many questions around that topic. Although they promote their LAN2Zigbee coordinator gateway (which is ok at <40 USD) I don't see why this should not also be possible with i.e. a Zigbee to WLAN bridge built with an ESP32 and a CC2530 and fed into HA through the ESPHome serial server component.

I will definitely do some testing in this realm over the next few weeks.

[Char-r]

Since we have serial2TCP option available now, we can think of a few scenarios that ZIGBEE2MQTT is handling more than a single meshed Zigbee network. (cf. adjacent buildings, different floors sharing the same wifi network, etc.)

serial:
    port: 'tcp://192.168.2.13:20108'

For me it's a 2 floors of reinforced concrete house, a single wifi network is done with ethernet cable connected APs, but no chance of Zigbee signals getting through - thick reinforced concrete walls and metal doors, wifi is not doing well, I had to install 4 wired APs - Not without using a few Zigbee routers set up along the walled/gated corridors and twisted stairs. (no power outlets along the stairs was the killing blow. btw, LAN cables are already installed in the wall)

I know CC26X2R might, yes might work, but I have several CC2531s and CC2530s lying around and wifi is already very reliable.

For now, I see that another instance of Docker container and cheap ESP01+cc2530 in each floor is the way to go. Before this, I had 2 Rpi's with cd2531+zigbee2mqtt running on each.

But still I wish zigbee2mqtt could handle multiple coordinators/networks.

@tanus10 were you ever able to find a workable solution? I am in you situation with a very old house and huge wall, can barely get the signal out of a single room so bought multiple network coordinators with the idea of having different networks for different areas of the house but then realised that Z2M could not handle multiple networks ...

[nickwild-999]

@Char-r I too have a massively thick wall in my old farmhouse. Did you find a solution to this?

[Jpsy]

While Z2M is not (yet) supporting multiple coordinators it is actually quite simple to join multiple Z2Ms on the MQTT level.

I have multiple Z2M instances connected to a Mosquito broker running on my Home Assistant server. And they play along perfectly with auto discovery of all entities into HA. This is a big advantage over ZHA on Home Assistant. ZHA also lacks the possibility to connect multiple coordinators but on top of that lacks any deeper communication level where you could join multiple instances.

IMHO MQTT is just the right point to do this. It is a broker protocol made to interconnect instances. And Z2M on a low profile hardware like RPi zero is just perfect for creating multiple Z2M hubs in such a setup.

[nickwild-999]

@Jpsy I am running a supervised installation of HA on a virtualbox on a macmini - I presume I would install Z2M on a rpi in one of the areas I am having difficulty in reaching?

[Jpsy]

Yepp. Correct. I.e. one RPi zero with Z2M and a ZigBee stick per level of your house. If you also need ZigBee where your HA is you can run one Z2M instance directly on HA as an addon. This is exactly my setup.

[nickwild-999]

Perfect thanks so much @Jpsy

[ascheucher]

@Jpsy do you use the same zigbee2mqtt topic for all the Z2M instances? And do you run the ZigBee networks on different channels as well?

[grmocg]

For what it is worth, I too am interested in multiple coordinators.

[sefininio]

I don't see a problem with running multiple RPis, each running a Z2M docker container with a Zigbee dongle connected to it, and using a different mqtt base topic & different channel, assuming they all publish to the same homeassistant integration topic. Moreover, far-away coordinators can probably reuse the same channel without disturbances.

For example:

1. Basement coordinator: base topic z2m-basement, HA topic homeassistant, channel 20
2. Ground coordinator: base topic z2m-ground, HA topic homeassistant, channel 11
3. TopFloor coordinator: base topic z2m-top, HA topic homeassistant, channel 20
4. Shed coordinator: base topic z2m-shed, HA topic homeassistant, channel 15

Since the basement and top floor are farthest away, they can reuse the same channel safely.

[swinster]

FWIW, I created multiple Z2M docker containers in a stack (also consisting of HA Docker, Mosquito and others) with some Pi v1 running SONOFF coordinators remotely connected via TCP/IP. This seems to work, albeit I made a stupid mistake (so don't repeat it).

When I copied the docker-compose configuration, in which I updated the container's name and the mapped port, it forgot to update the mapped volumes. This kind of worked, but when I removed devices from the second container, I, of course, destroyed the first container config as both pointed to the same data. 🤦. Make sure you setup a separate folder for the container volume!

[Jens-Wymeersch]

@sefininio can you please give me a view how one of your coordinator configs looks like in yaml (I'm mostly interested in the HA topic) ? Thank you so much

[sefininio]

@sefininio can you please give me a view how one of your coordinator configs looks like in yaml (I'm mostly interested in the HA topic) ? Thank you so much

Exactly like I described above

homeassistant: true
permit_join: false
frontend: true
mqtt:
  base_topic: z2m-family
  server: mqtt://IP:1883
  keepalive: 60
  reject_unauthorized: true
  version: 4

[Jens-Wymeersch]

@sefininio thank you for your quick response. You don't need a different pan_id ?

[sefininio]

@sefininio thank you for your quick response. You don't need a different pan_id ?

I don't have pan_id in my config, so I guess not 🤷