Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REQUEST] ESPEasy firmware on Sonoff ZBBridge (Sonoff Zigbee Bridge) #3021

Open
Hedda opened this issue Apr 16, 2020 · 46 comments
Open

[REQUEST] ESPEasy firmware on Sonoff ZBBridge (Sonoff Zigbee Bridge) #3021

Hedda opened this issue Apr 16, 2020 · 46 comments
Labels
Category: Controller Related to interaction with other platforms Status: Needs Info Needs more info before action can be taken Type: Discussion Open ended discussion (compared to specific question) Type: Feature Request Add a completely new feature (e.g. controller/plugin)

Comments

@Hedda
Copy link

Hedda commented Apr 16, 2020

Please consider porting ESPEasy to the Sonoff ZBBridge a (new Sonoff Zigbee Bridge by Itead).

https://www.itead.cc/sonoff-zbbridge.html

Itead has just launched this Sonoff ZBBridge as an inexpensive Sonoff Zigbee Bridge which is based on a similar design as the "Sonoff RF Bridge" (which several other alternative open-source ESP8266 firmware already has support for), this Sonoff ZBBridge is also ESP8266 base, like the Sonoff RF Bridge.

https://www.cnx-software.com/2020/04/16/sonoff-zbbridge-wifi-to-zigbee-gateway/

According to the teardown on notenoughtech.com it sounds as if it is based on ESP8266 (ESP8266EX) for WiFi and bridge/gateway/controller software and then uses a Silicon Labs EFR32MG21 (EFR32 Mighty Gecko) for Zigbee 3.0 radio module support via UART/serial interface:

https://notenoughtech.com/home-automation/sonoff/sonoff-zigbee-bridge-preview/

From the images, it even looks like they are reusing the same injection moulds as for their Sonoff RF Bridge (Sonoff RF Bridge 433) housing/enclosure/chassis appliance box:

https://sonoff.tech/product/accessories/433-rf-bridge

https://www.itead.cc/sonoff-rf-bridge-433.html

https://www.itead.cc/wiki/Sonoff_RF_Bridge_433

Tasmota does something similar with Zigbee2Tasmota but by connecting an ESP8266 to a Texas Instrument CC2530 module instead however it too is using a serial communication protocol, see:

https://tasmota.github.io/docs/Zigbee/

EZSP (EmberZNet Serial Protocol) interface that Silicon Labs uses is also well documented and already used by open source projects, see example Home Assistant's ZHA integration component via zigpy and bellows:

Z Smart System also has a Java device driver for Ember based serial dongles (among other dongles):

For reference, Z Smart System also has a sniffer library written in Java which uses same serial interface:

@Hedda Hedda changed the title [REQUEST] ESPEasy firmware on the Sonoff Zigbee Bridge (Sonoff ZBBridge) [REQUEST] ESPEasy firmware on Sonoff ZBBridge (Sonoff Zigbee Bridge) Apr 23, 2020
@Hedda
Copy link
Author

Hedda commented Apr 23, 2020

To get the equivalent SMD module from Silicon Labs then might want to look at EFR32MG1B Series 1.

Almost any ESP8266 development board (like a NodeMCU or a Wemos D1 Mini) with a Silicon Labs EFR32MG based Zigbee module like the Ebyte E180-ZG120B would just about be about the equivalent to having a hacked Sonoff ZBBridge, so such a setup could be used as a development environment.

The downside to using such setup instead of Sonoff ZBBridge is that the E180-ZG120B module will likely not come preloaded with firmware so have to build EmberZNet PRO Zigbee Stack firmware with coordinator device type config and flash firmware to the E180-ZG120B module first.

See a longer discussion about building Silabs EmberZNet firmware here -> xoseperez/espurna#2224

Ebyte's E180-ZG120A and E180-ZG120B are Zigbee 3.0 module based on EFR32MG1B SoC:

EFR32MG1B Series 1 is a 20dbm powerful Zigbee 3.0 radio capable SoC / chip in Silicon Labs EFR32 family which I understand also used the Ember / EZSP interface so could be made compatible with zipy's bellows radio library if flashed with the right firmware? So I guess main question is which exact firmware to use?

E180-ZG120B (and the previous E180-ZG120A) modules are sold on eBay and Aliexpress at low prices.

Example:

Ebyte also have this inexpensive development board called "E180-ZG120B-TB" based on that module:

This development is sold for less than $9 US-dollar on Aliexpress or about twice that on eBay UK:

Looks like you can just remove some jumpers to disable the USB converter to use the serial directly.

Again see bellows for zigpy's Silicon Labs library https://github.com/zigpy/bellows

@Hedda
Copy link
Author

Hedda commented Apr 28, 2020

FYI @SillyDay don't own a E180-ZG120B but has tried to build a firmware for it as per discussion here:

Koenkk/zigbee-herdsman#168

SillyDay GitHub repo for EFR32 firmware:

https://github.com/SillyDay/EFR32

@Hedda
Copy link
Author

Hedda commented May 5, 2020

FYI, as per related request https://github.com/zigpy/zigpy/issues/405 I just learned that there is an existing ESP8266-based networked-attached Zigbee-adapter called "ZiGate Pack WiFi adapter" which has a new v2.0 firmware that archives this requested function of UART-to-TCP/IP (for Serial-port to WiFi-bridge function) using "ESP-LINK from Jeelab" software, in addition, using ESP-LINK also adds mDNS zeroconf to allow automatic network discovery and configuration:

Description of functions that using ESP-LINK will add to v2.0 firmware for ZiGate Pack WiFi adapter:

https://translate.google.com/translate?sl=fr&tl=en&u=https%3A%2F%2Fzigate.fr%2Fdocumentation%2Fdescription-du-firmware-v2-xx%2F

  • ZiGate TCP 9999 to UART WiFi Bridge
  • WiFi configuration in client and access point mode
  • DHCP client or static IP
  • Client syslog
  • SNTP client
  • mDNS
  • MQTT client
  • WiFi firmware update (OTA)
  • Console for logging and validating the operation of the ZiGate

As I understand it, all ZiGate hardware look to be modular in design and the "ZiGate Pack WiFi adapter" is really just an optional ESP8266 based "dumb" UART-to-TCP/IP (for Serial-port to WiFi-bridge function) for the standard "ZiGate TTL adapter" that allows users to connect to it remotely using TCP/IP over your home LAN (Local Area Network) instead of plugging it directly to your computer via the optional USB adapter.

Specifically, please see the picture of "ZiGate Pack WiFi adapter" https://zigate.fr/produit/zigate-pack-wifi-v1-3/ compared to the picture of "ZiGate TTL USB adapter" https://zigate.fr/produit/zigate-ttl/

Thus "ZiGate Pack WiFi adapter" allows ZHA users to have a networked Zigbee adapter setup like this:

ZHA <–> zigpy/zigpy-zigate <–> TCP/IP over LAN <–> ZiGate-WiFi <–> UART <–> ZiGate Radio

Suggesting this now as I just learned from @doudz there that the new v2.0 version of the ZiGate Pack WiFi adapter firmware contains "ESP-LINK from Jeelab" software which among other things adds mDNS and UART WiFi Bridge support over TCP. As I understand, version v1.x of the firmware for the ZiGate Pack WiFi adapter basically only contained a simple UART/serial-port server forwarding service ( serial server software that just acts as a dumb Zigbee to WiFi bridge for zigpy-zigate), while the new version v2.x also has more advanced features (which does not need to used) it still also contain a simple UART/serial-port server forwarding service, but now mDNS also makes it easier to discover the adapter on your local network.

Now it would be awesome if the ZHA integration component for Home Assistant from an end-user perspective supported just as an easy detection and configuration of network networked-attached Zigbee coordinator adapters, like the Sonoff ZBBridge and the ZiGate Pack WiFi adapter.

I would therefore also suggest using some kind of Zero-configuration networking (zeroconf) method, like for example mDNS, (as mDNS is already in use in Home Assistant Core), to make the ZHA integration component for Home Assistant automatically detect, connect, and configure compatible networked-attached Zigbee coordinator adapters like the "ZiGate Pack WiFi adapter" as that is otherwise already supported by the zigpy-zigate radio library for zigpy.

@TD-er
Copy link
Member

TD-er commented May 5, 2020

FYI: mDNS is also present in ESPEasy, but apparently it was recently broken in the esp8266/Arduino core (lots of issues recently added there)
It is not enabled in all builds as it does take extra resources and we're already running low on those.

I agree it would be a nice feature to have Zigbee support in ESPEasy.
But to be honest, I don't know how much time I will have to implement support for it in the next weeks/months.

@uzi18
Copy link
Contributor

uzi18 commented May 5, 2020

there is alternative in PRs of core, but maybe some modifications are needed we can try it
will post a link later

@Hedda
Copy link
Author

Hedda commented May 5, 2020

If we could hack the new Sonoff ZBBridge to run ESPEasy firmware on its ESP8266 then a workaround could be to use some kinda UART-to-TCP/IP (for Serial-port to WiFi-bridge function) service to forward the serial communication of the Zigbee radio module to a remote computer running zigpy, like for the ZHA integration component built-into Home Assistant.

That is, that would allow users to use a networked Sonoff ZBBridge in the best location in their home for radio reception instead of using local adapter connected directly to the computer running Home Assistant with ZHA integration component for Zigbee.

@uzi18
Copy link
Contributor

uzi18 commented May 5, 2020

@TD-er this one esp8266/Arduino#6871

@uzi18
Copy link
Contributor

uzi18 commented May 5, 2020

about esp-link know this project as also supported them with bug fixes and new features
will look later how far this project is from original esp-link

@uzi18
Copy link
Contributor

uzi18 commented May 5, 2020

@Hedda BTW espeasy already have support for ser2net aka Serial-port to WiFi-bridge

@Hedda
Copy link
Author

Hedda commented May 5, 2020

So if you could add mDNS support on top of that to ESPEasy as well as have the support for it in zigpy and bellows then a Sonoff ZBBridge hacked to run ESPEasy could possibly become the most awesome and inexpensive network-attached Zigbee adapter for Home Assistant's ZHA integration component.

Again I suggest to checkout Home Assistant's ZHA integration component use of zigpy and bellows

ZHA does today not have support for ser2net or socat in its configuration flow, nor mDNS support.

Would be great if the ZHA side (client-side) of it could be made as user-friendly experience as possible.

@Barracuda09
Copy link

Could this be the same as the Ikea TRÅDFRI Gateway?

@Hedda
Copy link
Author

Hedda commented May 6, 2020

@Barracuda09 Yes could replace proprietary Zigbee hubs/gateways/bridges such as IKEA TRADFRI Gateway, Philips Hue Bridge, Osram Lightify Gateway, and Xiaomi Aqara Gateway if that is what you are asking. You would still need a home automation software like Home Assistant, OpenHAB, Domoticz, etc. to act as your front-end and for automations.

@Barracuda09
Copy link

@Hedda

Well I mean that the Ikea TRÅDFRI Gateway has the same CPU

@Hedda
Copy link
Author

Hedda commented May 7, 2020

@Barracuda09 CPU? Not sure what you mean by "CPU" in reference to this request. The IKEA TRADFRI Gateway is not based on ESP8266/ESP8285 or ESP32 if that is what you mean.

IKEA TRADFRI Gateway does use a Silicon Labs Mighty Gecko EFR32MG1 Series 1 (EFR32MG1P132GI) but it is not based on ESP8266/ESP8285 or ESP32 so not really relative to this request or to ESPEasy. That is unless you mean to hack the IKEA TRADFRI Gateway by soldering an ESP8266/ESP8285 or ESP32 module to it only to be able to use its Zigbee radio module.

See https://www.ifixit.com/Teardown/Ikea+TR%C3%85DFRI+Gateway+Teardown/85936

Other using a similar Silabs EFR32 "Ember" family chip for the Zigbee 3.0 radio module the IKEA TRADFRI Gateway it can not be hacked to run ESPEasy because doesn't have a ESP8266/ESP8285 or ESP32 that runs the "gateway/bridge/hub" application software.

These type of gateways are usually based on several SoC MCU ( System-on-a-Chip Microcrontroller Unit) modules/chips and yes those MCUs, in turn, each contains some kind of CPU or CPUs, but which "CPU" that is exactly is not really relative here.

These types of gateways are usually each based on two SoC MCU, where one of the SoC MCU is the Zigbee radio module/chip and the other SoC MCU runs the "gateway/bridge/hub" application software and has the WiFi and/or Ethernet function, though I believe that the IKEA TRADFRI Gateway runs the "gateway/bridge/hub" application software on its EFR32MG1P132GI module.

Some types of these gateways, can even use three or four SoC MCU if they have a separate module(s)/chip(s) for the WiFi and/or Ethernet functions. The IKEA TRADFRI Gateway in this example has a Murata Type1GC 2.4GHz & 5GHz Wi-FI Module ("module includes the Cypress BCM43907 Wireless System SoC") for WiFi and a Broadcom BCM5241 Fast Ethernet Transceiver for Ethernet (Wired Network).

@Barracuda09
Copy link

@Hedda

Sorry, I had not red your request well. Sonoff uses an ESP8266 (as you said) to 'interface' to the EFR32MG21 SOC.

I only saw EFR32.... and thought/red no further, and concluded that the Ikea was the same as Sonoff. But now reading your initial request it makes it clear for me.

@Hedda
Copy link
Author

Hedda commented Jun 1, 2020

FYI, @s-hadinger has now got one and started review arendst/Tasmota#8583

First signs it is the EFR32 Zigbee module it has inside it has a EmberZNet / Ember based firmware on it.

@Hedda
Copy link
Author

Hedda commented Jun 2, 2020

There is now also a deep dive follow-up discussion here in parallel about hacking or flashing its EFR32:

@Hedda
Copy link
Author

Hedda commented Jun 22, 2020

FYI; @s-hadinger is making great progress hacking the Zigbee module in Sonoff ZBBridge for Tasmota:

@mtx512 has now built a custom EZSP NCP firmware for the EFR32 module inside Sonoff ZBBridge:

Interestingly @s-hadinger also added a TCP serial bridge to Tasmota firmware for remote access:

Description:

Add a new feature: Serial to TCP bridge (similar to esp-link). This allows to remotely communicate to a MCU through ESP8266. Needs #define USE_TCP_BRIDGE

It adds 2 GPIO types: TCP Tx (208) and TCP Rx (209) and can work with hardware or software serial.

Commands:

  • TCPBaudRate <x>: sets the baudrate for serial (only 8N1 mode), min 1200, max 115200 by 1200 increments.
  • TCPStart <port>: listens to port <port>. This features supports 2 parallel TCP connexions, which can be useful if you need a terminal + a specific protocol (like XMODEM). The 3rd connection will disconnect an previous connection. The number of parallel connections is a compile-time option.
  • TCPStart 0 or TCPStart: shuts down the TCP server and disconnects any existing connection.

For security reasons, the TCP bridge is not started at boot, and requires an explicit TCPStart command (can be automated with Rules).

@Hedda
Copy link
Author

Hedda commented Jun 23, 2020

FYI, Silicon Labs recently just released a less expensive "EFR32xG22 Wireless Gecko Starter Kit" (SLWSTK6021A) development kit for $99 which only contain one mainboard/development board and a couple of EFR32MG22 modules which are low-power Zigbee Green Power compatible Mightly Gecko Series 2 radios.

https://www.silabs.com/products/development-tools/wireless/efr32xg22-wireless-starter-kit

While the EFR32MG22 low-power radios it comes with are not really usable as Zigbee coordinators, for $99 that kit can be very useful to developers since it jas been confirmed that buying that kit also gives full official access to Silicon Labs Zigbee Stack, Zigbee SDK, and libraries (current, future, and old versions).

@TD-er
Copy link
Member

TD-er commented Jun 23, 2020

While the EFR32MG22 low-power radios it comes with are not really usable as Zigbee coordinators, for $99 that kit can be very useful to developers since it jas been confirmed that buying that kit also gives full official access to Silicon Labs Zigbee Stack, Zigbee SDK, and libraries (current, future, and old versions).

Hmm that last part suggests it may not be freely available to release such a stack in an open source project.

@mtx512
Copy link

mtx512 commented Jun 23, 2020

While the EFR32MG22 low-power radios it comes with are not really usable as Zigbee coordinators, for $99 that kit can be very useful to developers since it jas been confirmed that buying that kit also gives full official access to Silicon Labs Zigbee Stack, Zigbee SDK, and libraries (current, future, and old versions).

Hmm that last part suggests it may not be freely available to release such a stack in an open source project.

Yes, the SL licence agreement states only binary code can be distributed if code is generated from the SDK.

@Hedda
Copy link
Author

Hedda commented Jun 23, 2020

While the EFR32MG22 low-power radios it comes with are not really usable as Zigbee coordinators, for $99 that kit can be very useful to developers since it jas been confirmed that buying that kit also gives full official access to Silicon Labs Zigbee Stack, Zigbee SDK, and libraries (current, future, and old versions).

Hmm that last part suggests it may not be freely available to release such a stack in an open source project.

As noted.; that binary code (firmware images) can be freely distributed if code is generated from SDK.

Note that Silicon Labs Zigbee stack is not directly incorporated into ESPEasy or any other open source project, the SDK is only for building firmware to run on the Silicon Labs MCU. To integrate it you only use serial communication to talk to it using open APIs or CLIs which have public documentation available.

ESP8266 with ESPEasy serial bridge <-> EFR32 with EmberZNet NCP firmware

However, for ESPEasy is might be most relevant to only offer the same multi-port TCP serial bridge that Tasmota now has to just act as a serial server for the Silabs MCU so that it can be served to another computer on the network which does actual communication with the Silicon Labs Zigbee stack. (This is the same for all and any propitiatory Zigbee stack from any manufacturer).

arendst/Tasmota#8702

bellows (zigpy) <-> ESP8266 with ESPEasy serial bridge <-> UART/Serial <-> EFR32 with EmberZNet NCP firmware

Home Assistant ZHA <-> ESP8266 with ESPEasy serial bridge <-> UART/Serial <-> EFR32 with EmberZNet NCP firmware

Domoticz Zigpy plugin <-> ESP8266 with ESPEasy serial bridge <-> UART/Serial <-> EFR32 with EmberZNet NCP firmware

openHAB Zigbee binding <-> ESP8266 with ESPEasy serial bridge <-> UART/Serial <-> EFR32 with EmberZNet NCP firmware

zigbee-herdsman <-> ESP8266 with ESPEasy serial bridge <-> UART/Serial <-> EFR32 with EmberZNet NCP firmware

Notice that Tasmota's new TCP serial bridge features support for 2 parallel TCP connexions, which can be useful if you need a terminal + a specific protocol (like XMODEM).

arendst/Tasmota#8702

@TD-er
Copy link
Member

TD-er commented Jun 23, 2020

That looks simple enough.
We also support the I2C UART, so it may be easier to add it to any node as you're not running out of GPIO pins.

@Hedda
Copy link
Author

Hedda commented Jun 24, 2020

@TD-er regardless I recommend that you also check out the related and interesting progress being made to get Tasmota firmware on the ESP8266 inside the Sonoff ZBBridge (Sonoff Zigbee Bridge)

arendst/Tasmota#8583

Tasmota developers is planning on getting Tasmota firmware running on the ESP8266 inside the Sonoff ZBBridge and then both letting other third-party software access the Silabs MCU for as a remote Zigbee adapter over that TCP serial bridge as an option as well also getting Tasmota to control that Silabs MCU directly itself to act as a Zigbee gateway via its Zigbee2Tasmota implementation/solution

https://tasmota.github.io/docs/Zigbee/

@Hedda
Copy link
Author

Hedda commented Jun 24, 2020

FYI, Silicon Labs recently just released a less expensive "EFR32xG22 Wireless Gecko Starter Kit" (SLWSTK6021A) development kit for $99 which only contain one mainboard/development board and a couple of EFR32MG22 modules which are low-power Zigbee Green Power compatible Mightly Gecko Series 2 radios.

https://www.silabs.com/products/development-tools/wireless/efr32xg22-wireless-starter-kit

While the EFR32MG22 low-power radios it comes with are not really usable as Zigbee coordinators, for $99 that kit can be very useful to developers since it has been confirmed that buying that kit also gives full official access to Silicon Labs Zigbee Stack, Zigbee SDK, and libraries (current, future, and old versions).

Those who are interested in that official $99 SLWSTK6021A started kit might also be interested in buying article SLWRB4180A for around $49 at the same time as that is the matching EFR32MG21 based high-power (+20 dBm) radio board for that dev board kit.

https://www.silabs.com/products/development-tools/wireless/slwrb4180a-efr32-wireless-gecko-radio-board

(Google search shows that SLWRB4180A can be bought from many of resellers globally who also stock the SLWSTK6021A kit).

@Hedda
Copy link
Author

Hedda commented Jul 3, 2020

FYI, @s-hadinger has after the last pull request posted saying that he now feels Tasmota ready for testing of EZSP v8 interface:

arendst/Tasmota#8583

arendst/Tasmota#8839

Not for everyone yet since still need to flash its EFR32 chip with some type of J-Link Debug Probe (might always be required?).

@Hedda
Copy link
Author

Hedda commented Sep 11, 2020

Any further thoughts on Sonoff ZBBridge support now that this is fully supported and already very stable feature in Tasmota?

arendst/Tasmota#8583

That Sonoff ZBBridge is inexpensive but contain great Zigbee hardware with potential for different types of Zigbee controller uses, as inside it has an ESP8266 WiFi SoC/MCU connected via UART to a very powerful Silicon Labs EFR32MG2 (Mighty Gecko Series 2) Zigbee SoC/MCU on the same board.

https://sonoff.tech/product/smart-home-security/zbbridge

Tasmota allows using its ser2net like TCP serial-server inside ESP8266 firmware to take direct control of the Zigbee chip that it is connected to via UART/serial and then connect to Zigbee devices without any further involvement on Tasmota (which via its TCP serial-server only will only act as a dumb pass-through WiFi-to-serial bridge without any knowledge of what the data traffic that is going back and forth between the remote Zigbee chip AND different home automation software like example Home Assistant).

You can even make your own DIY WiFi bridge with an ESP8266 board + an ICC-A-1 module torn out of an IKEA Tradfri device:

https://github.com/MattWestb/IKEA-TRADFRI-ICC-A-1-Module

Example with ZHA integration component in Home Assistant which add native support for Zigbee adapters, including the ZBBridge:

https://www.digiblur.com/2020/07/how-to-use-sonoff-zigbee-bridge-with.html

Sure, it could have been even greater with ESP32 but still geat ESP8266 hardware with Silabs EFR32 MG2 Zigbee coordinator!

@TD-er
Copy link
Member

TD-er commented Sep 11, 2020

Any further thoughts on Sonoff ZBBridge support now that this is fully supported and already very stable feature in Tasmota?

Not yet.
Not saying it will not be included in ESPEasy also, but right now I'm working full time on making ESPEasy as stable as can be with all the things we've currently included.
Also I don't have Z-wave/Zigbee devices, so it is not yet possible for me to test with it.
And right now we're also constantly running into build (size) issues, so adding more will not improve that.

If you want to make a PR, I am more than happy to have a look at it, but right now I simply don't have the time for it.

@Hedda
Copy link
Author

Hedda commented Sep 11, 2020

Also I don't have Z-wave/Zigbee devices, so it is not yet possible for me to test with it.

IMHO the important part of the request is for ESPEasy to have some kind of integrated TCP serial-server (transparent Serial/UART pass-through/relaying) as a network bridge/gateway, a networked-attached serial-port, that is stable up to at least 256 Kbps bit rate.

An example of how you normally use ser2net and socat to connect to remote serial port over TCP/IP

https://www.mankier.com/8/ser2net

http://www.dest-unreach.org/socat/

Does not really have to be used and tested specifically with RF serial modules/adapters like Zigbee, Z-Wave, Thread, Bluetooth Low Energy (BLE), LoRa, 6LoWPAN, Sigfox, or other proprietary RF (315MHz/433MHz/868MHz/915MHz) modules/adapters like RFLink, for which I agree there are all a good reason to have serial to network bridge, as one should really be able to use and test the TCP serial server bridge to proxy any Serial/UART/RS-232 device.

Virtual serial port redirection could be used and tested with any serial device or COM port attached peripheral. For home automation, there are a lot of non-RF devices that have a serial communication interface, such as sensors (like example MySensors), IR/infrared, alarm interfaces, GPS/GPRS receivers, generic serial communication, etc.

@Hedda
Copy link
Author

Hedda commented Mar 11, 2021

FYI, HW tip now is to instead use Tube's Zigbee Gateways open-source hardware by @tube0013 (which is tested with ESPHome).

It is based on WT32-ETH01 ESP32 board from Wireless-Tag which also has wired Ethernet.

He designed two variants; one has a Silicon Labs EFR32 Series 2 module and one that has a Texas Instruments CC2652P module.

https://github.com/tube0013/tube_gateways

zigpy/zigpy#584

https://www.tubeszb.com/shop/coordinators/2

image

image

@Barracuda09
Copy link

Barracuda09 commented Mar 26, 2021

Hi @ALL

I have done some test with Sonoff ZBBridge runing with Tasmota in combination with Amazones Alexa.

Which turned out to be a challenge to make it work with original code (had some bugs) and after the update of Alexa it again was a challenge to make it work again (the original Tasmota fix did not do the trick).

What would be the challenge to make it work in ESPEasy @TD-er ?

Edit:
So the test did include the Philips Hue Emulation which was the challenge to make it work

Regards,
Marc

@TD-er
Copy link
Member

TD-er commented Mar 26, 2021

What would be the challenge to make it work in ESPEasy @TD-er ?

At this moment the most important missing ingredient is time.

@Barracuda09
Copy link

Hi @TD-er

Ok, I understand. But what if I could pick up the glove, where should I start? Do you have some pointers?

Or should I just dive into it, I have not (yet) much experince in the build enviroment.

Kind regards,
Marc

@TD-er
Copy link
Member

TD-er commented Mar 26, 2021

What is exactly needed to communicate with the Zigbee module?
Also I'm not entirely sure what this should be.

Is it a controller (sending out data) or is it a plugin (receiving data)?
And what information is received? Is it some numerical value, or is it a tree-like structure with a value (e.g. similar to a topic + message of MQTT)

What kind of data should be sent?

If it is too complex to map directly to existing ESPEasy "object types" then I wonder what it should be.
Should support mean ESPEasy is a proxy between some platform (e.g. OpenHAB/HomeAssistant) and a serial connected radio module?

I imagine the Zigbee nodes are addressed via some address (similar to a MAC address?) and given a data structure containing commands.
This suggests support should be in the form of a controller. But how do we set a specific address and data structure format to a task configuration? (e.g. Domoticz needs an IDX value, which can be set to a task configuration, but that's only 32 bit)
Such an IDX value can of course be used as some kind of alias/index in extended controller configuration.

If that becomes too complex (like: needs a lot of changes in the settings structure) then it may be best to just add support via commands to be executed from rules.

When it is about receiving data, I guess the most flexible way is to trigger events to be handled in rules.
But then we must know what format is received as string handling in rules is not yet working everywhere. (e.g. you can have a string in an event value, but not match on it in the on eventname=sometext do rules block.

@Hedda
Copy link
Author

Hedda commented Mar 26, 2021

But what if I could pick up the glove, where should I start? Do you have some pointers?

Hope that you understand that is would be a huge effort difference between putting the whole Zigbee gateway/bridge application-layer inside the ESPEasy firmware (following the concept that Zigbee2Tasmota/Z2T project have adopted) compared to only acting as a serial-to-IP proxy-server to enable "remote serial communication access" to external third-party applications (like Home Assistant, OpenHAB, or Zigbee2MQTT) that is running on other computers like a Raspberry Pi.

IMHO it would be great just to have the perfect proxy-server for external third-party application which enhanced features such as plug-and-play / auto-detection functions and encryption features for secure serial-to-IP communication. See zigpy/zigpy#667

If you however decide to go down the rabbit hole with putting a whole Zigbee gateway/bridge application inside the ESPEasy then please try to make it into modular libraries that could also be reused and shared by other ESP firmware projects. As other than Zigbee2Tasmota/Z2T project by @s-hadinger I also read that the developer of OpenMQTTGateway is maybe now also thinking about adding similar features. See comments by @1technophile in 1technophile/OpenMQTTGateway#205

It would be an interesting addition to OMG, but maybe the best way would be to externalize it into a Zigbee library. This way it will benefit the different projects (OMG, ESPurna, ESPEasy, ESPHome...).

Or do you mean splitting all the Zigbee code parts out on Zigbee2Tasmota (Z2T) into an Arduino library which could be reused?

Exactly. This way it could be used by other projects.

Also, as an alternative to using MQTT as the message protocol over IP Zigbee2Tasmota/Z2T project do you might like to look into the "Project Connected Home over IP Application Layer" instead of as a way for the gateway/bridge protocol to communicate and integrate with other home automation software:

https://github.com/project-chip/connectedhomeip

https://www.connectedhomeip.com

PS: If you choose to go on a similar route as Zigbee2Tasmota/Z2T project then be aware that will really be limited to ESP32 devices.

@Hedda
Copy link
Author

Hedda commented Mar 26, 2021

What is exactly needed to communicate with the Zigbee module?

A tip would be to look at both the adapter code from zigbee-herdsman as well as the different radio libraries for zigpy projects:

Another recommendation is begin with Texas Intruments Z-Stack 3 ZNP based Zigbee controllers such as CC1352P and CC2652P:

PS: CC1352 is cost more but allow Sub-1 GHz frequencies used by Zigbee Smart Energy (ZSE) profile which isn't part of Zigbee 3.0

@TD-er
Copy link
Member

TD-er commented Mar 26, 2021

PPS: If you choose to go on a similar route as Zigbee2Tasmota/Z2T project then be aware that will really be limited to ESP32 devices.

Not sure if that's really a drawback here.
Not saying ESP8266 is some kind of EOL, absolutely not!
But given people may need to create devices like the one you showed and it seems very likely people want to connect it to ethernet to get a more stable network connection and perhaps the option for PoE to limit the number of wires, then it makes only sense to use ESP32 and not force yourself into ESP8266-support.
Also the ESP32 does have 3 hardware UARTs on board, which is preferrable for interfacing with such a module.
Given the price of ESP32 has dropped to ESP8266 levels, then I don't see why newly designed boxes for this purpose will be built around ESP8266.
I have no idea how much memory is needed for a Zigbee stack, but if you need to implement that also on the ESP, then I guess it is a no-brainer to go for ESP32 only for this as those have enough free RAM for such stacks.

@TD-er TD-er added Category: Controller Related to interaction with other platforms Status: Needs Info Needs more info before action can be taken Type: Discussion Open ended discussion (compared to specific question) Type: Feature Request Add a completely new feature (e.g. controller/plugin) labels Mar 26, 2021
@Hedda
Copy link
Author

Hedda commented Mar 26, 2021

I have no idea how much memory is needed for a Zigbee stack, but if you need to implement that also on the ESP, then I guess it is a no-brainer to go for ESP32 only for this as those have enough free RAM for such stacks.

Zigbee2Tasmota/Z2T project actually runs on ESP8266 today but I understand that its resource constraints are limiting the features and functions that they might want can add, and I believe that they are in the process of porting it to Tasmota32 for ESP32 support.

The Zigbee stack runs on the Zigbee MCU which are relatively powerful wireless chips in their own right, (see SoC examples like CC1352P, CC2652P, EFR32MG21, EFR32MG12, and EFR32MG13), but to create a stand-alone Zigbee gateway/bridge that runs natively on the ESP32 it will need to run ZCL (Zigbee Cluster Library), ZDO (Zigbee Device Object), for application state management and a database for all the Zigbee devices.

Remember that using a firmware like ESPEasy as a Zigbee gateway/bridge would in theory enable users to connect hundreds of Zigbee devices that need to have features for discovering, configuring, maintaining and state tracking.

Checkout
https://training.ti.com/
https://training.ti.com/what-zigbee-introduction-and-look-tis-solutions
https://www.ti.com/wireless-connectivity/zigbee/overview.html
https://github.com/SiliconLabs/IoT-Developer-Boot-Camp
https://github.com/SiliconLabs/IoT-Developer-Boot-Camp/wiki

As a proof-of-concept see zigpy as an example https://github.com/zigpy/zigpy

@TD-er
Copy link
Member

TD-er commented Mar 26, 2021

Already thought it might be something like that.
So for sure a really good interface to support, similar to the usefulness of WiFi/Ethernet. But that also means you simply can't implement it all at once as that will be a never ending story and in essence a separate project alongside ESPEasy for the development duration.
This means the implementation should be split into several stages each with their own deliverables.

If you know a short introduction (e.g. high level introduction video) for me to get an idea of what it all entails then we can think of a way to split it into parts so that we know what functional blocks of ESPEasy can be mapped onto what concepts of Zigbee and also what needs to be changed or made.

@Barracuda09
Copy link

Hi @Hedda and @TD-er,

You guys have already put in a lot of thought into this subject, as I only have scratch the surface of it all.
There is a lot of information here to investigate for me. And I am not sure what this all means just now.

I use the Sonoff ZBBridge with Hue Emulation to control the Zigbee Lights with Alexa.


I have tested switching with MQTT, with something like:

  • Payload: { "device":"WallSwitch", "send":{"Power":"On"} }
  • topic: cmnd/tasmota_E2AE3D/ZbSend

And read-in some temperature and humidity sensors into Domoticz.

@TD-er : the memory usage
image

@Hedda
Copy link
Author

Hedda commented Mar 26, 2021

So for sure a really good interface to support, similar to the usefulness of WiFi/Ethernet. But that also means you simply can't implement it all at once as that will be a never-ending story and in essence a separate project alongside ESPEasy for the development duration.

Never-ending story indeed! I would guess implementing a native Zigbee gateway/bridge solution similar to the Zigbee2Tasmota (Z2T) project would be exactly that; "in essence a separate project alongside ESPEasy for the development duration".

You also need to understand that not only are they many different types of Zigbee device specification but no all Zigbee devices manufacturers follow the standard Zigbee specifications, (especially if they are just simple lighting devices), which means you in a worst-case scenario need to create device converters/handlers which convert/translate any non-standard device information/messages into information/messages that do follow the standard Zigbee specifications. Projects like zigbee-herdsman (which Zigbee2MQTT and IoBroker depends on) and zigpy (which Home Assistant's ZHA integration and Jeedom Zigbee plugin depends on) solve this by having a translation library that basically overwrites the deviating and non-conforming information/messages that comes from selected devices that do follow the standards. So not to reinvert the wheel completely it would probably be best if you too could rely one on of those existing libraires for translations:

https://github.com/zigpy/zha-device-handlers

https://github.com/koenkk/zigbee-herdsman-converters

@Barracuda09 To clearify; that picture you posted show that you are Tasmota firmware with the Zigbee2Tasmota/Z2T project (and the Hue Emulation for Alexa is only one of the features that the Zigbee2Tasmota/Z2T supports):

https://tasmota.github.io/docs/Zigbee/

https://tasmota.github.io/docs/Zigbee-Internals/

If you know a short introduction (e.g. high level introduction video) for me to get an idea of what it all entails then we can think of a way to split it into parts so that we know what functional blocks of ESPEasy can be mapped onto what concepts of Zigbee and also what needs to be changed or made.

Texas Instruments and Silicon Labs both have high-level introduction/educationals videos to building Zigbee implementations.

https://training.ti.com/what-zigbee-introduction-and-look-tis-solutions

https://www.silabs.com/support/training/zigbee-software-bootcamp/zigbee-preparatory-course

If you guys are thinking of implementing something like the Zigbee2Tasmota (Z2T) project then obviously looking at their documentation and code is a must, however, you might also want to consider looking at the internal architecture of the Zigbee2MQTT project and its dependencies as I understand much the high-level concept for the Zigbee2Tasmota (Z2T) project was borrowed from the Zigbee2MQTT project, and I suspect that Zigbee2Tasmota (Z2T) project perhaps had to simply more maybe needed if they has chosen to only target ESP32 instead of initially only targeting ESP8266.

https://tasmota.github.io/docs/Zigbee/

https://tasmota.github.io/docs/Zigbee-Internals/

https://www.zigbee2mqtt.io/

https://github.com/Koenkk/zigbee2mqtt

https://github.com/koenkk/zigbee-herdsman

https://github.com/koenkk/zigbee-herdsman-converters

Internal Architecture

Zigbee2MQTT is made up of three modules, each developed in its own Github project. Starting from the hardware (adapter) and moving up; zigbee-herdsman connects to your Zigbee adapter an makes an API available to the higher levels of the stack. For e.g. Texas Instruments hardware, zigbee-herdsman uses the TI zStack monitoring and test API to communicate with the adapter. Zigbee-herdsman handles the core Zigbee communication. The module zigbee-herdsman-converters handles the mapping from individual device models to the Zigbee clusters they support. Zigbee clusters are the layers of the Zigbee protocol on top of the base protocol that define things like how lights, sensors and switches talk to each other over the Zigbee network. Finally, the Zigbee2MQTT module drives zigbee-herdsman and maps the zigbee messages to MQTT messages. Zigbee2MQTT also keeps track of the state of the system. It uses a database.db file to store this state; a text file with a JSON database of connected devices and their capabilities.

@Hedda
Copy link
Author

Hedda commented Sep 6, 2021

FYI, thegroove made a simple to use Serial-to-IP bridge appliance firmware for Sonoff Zigbee Bridge running ESPHome firmware:

https://github.com/thegroove/esphome-zbbridge

oxan and thegroove also look to have made two alternative serial server code component solutions for ESPHome firmware:

https://github.com/oxan/esphome-stream-server

https://github.com/thegroove/esphome-serial-server

thegroove even made an experimental version with Zeroconf support for automatic network discovery by ZHA in Home Assistant:

https://github.com/thegroove/esphome-zha-ezsp-zeroconf

https://www.home-assistant.io/integrations/zha/#discovery-via-usb-or-zeroconf

Note that Zigbee2MQTT is another popular alternative to Home Assistant's ZHA for use with this type of Serial-to-IP bridge:

https://www.digiblur.com/2021/03/zigbee2mqtt-with-sonoff-zigbee-bridge.html

https://www.digiblur.com/2020/07/how-to-use-sonoff-zigbee-bridge-with.html

@Hedda
Copy link
Author

Hedda commented Sep 22, 2021

FYI, ZB-GW03 eWeLink Ethernet Zigbee Gateway is based on an ESP32 and Silabs EFR32 Zigbee and has been hacked with Tasmota.

arendst/Tasmota#12764

ZB-GW03 has been hacked with Tasmota32 and look to support the same Tasmota and Silabs firmware as ITead Sonoff ZBBridge, however since the ZB-GW03 has wired Ethernet port instead of WiFi it should be a better option compared to ITead Sonoff ZBBridge.

https://templates.blakadder.com/ewelink_ZB-GW03

image

image

image

ZB-GW03 v1.0 and ZB-GW03-V1.2 is apparently sold rebranded under many names, including SmartWise and EACHEN brands:

https://www.okosabbotthon.hu/smartwise-zigbee-bridge-pro

https://www.amazon.de/APP-Fernbedienung-Sprachsteuerung-Funktioniert-Verbindung-Smart-Produkten/dp/B094JKVLNR/

image

image

Looks like is it is based on the same "SM-011 V1.0" module by CoolKit as ITead uses in their Sonoff ZBBridge Zigbee Bridge:

zigpy/zigpy#586

https://www.coolkit.cn/product/sm-011/

https://github.com/CoolKit-Technologies/DevDocs/tree/master/Zigbee

https://github.com/CoolKit-Technologies/DevDocs/blob/master/Zigbee/SM-011%E5%BA%94%E7%94%A8%E6%8C%87%E5%AF%BC%E4%B9%A6.md

https://translate.google.com/translate?hl=&sl=auto&tl=en&u=https%3A%2F%2Fgithub.com%2FCoolKit-Technologies%2FDevDocs%2Fblob%2Fmaster%2FZigbee%2FSM-011%E5%BA%94%E7%94%A8%E6%8C%87%E5%AF%BC%E4%B9%A6.md

@TD-er
Copy link
Member

TD-er commented Sep 22, 2021

Looks interesting.
Too bad they did not include PoE.

@Hedda
Copy link
Author

Hedda commented Sep 22, 2021

Looks interesting.
Too bad they did not include PoE.

If need PoE and just want to buy then check out Tube's CC2652P2 Based Zigbee to Power Over Ethernet Serial Coordinator V2

https://www.tubeszb.com/product/cc2652_poe_coordinator/21?cp=true&sa=false&sbp=false&q=false&category_id=2

Board is based on Olimex ESP32-POE-ISO running ESPHome firmware + Tube has also released it as open source hardware:

https://github.com/tube0013/tube_gateways

Again it uses ESP32-PoE is an IoT WIFI/BLE/Ethernet development board and a Zigbee radio module with UART interface:

https://www.olimex.com/Products/IoT/ESP32/ESP32-POE-ISO/open-source-hardware

An alternative ESP32 PoE board is to go with LILYGO TTGO T-Internet-POE

http://www.lilygo.cn/prod_view.aspx?TypeId=50033&Id=1307

If building your own then base it on a Silabs MGM210P module (MGM210PA32JIA or MGM210PB32JIA) with Zigbee firmware:

https://www.silabs.com/wireless/zigbee/efr32mg21-series-2-modules

https://github.com/grobasoz/zigbee-firmware/

https://github.com/zha-ng/EZSP-Firmware

The most popular Zigbee radio modules for DIY projects are otherwise RFSTAR RF-BM-2652P2 and Ebyte E72-2G4M20S1E:

https://github.com/Koenkk/Z-Stack-firmware/blob/master/coordinator/Z-Stack_3.x.0/bin/README.md

https://github.com/Koenkk/Z-Stack-firmware/tree/master/coordinator

Tube has both models with both Silabs and Texas Instruments Zigbee modules but currently, the PoE model is TI Zigbee only.

@Hedda
Copy link
Author

Hedda commented Nov 1, 2021

FYI, there's now a detailed step-by-step instruction about flashing Tasmota on ZB-GW03 eWeLink Ethernet Zigbee Gateway (wired network bridge) for Zigbee2Tasmota (also using with Home Assistant's ZHA):

https://thehelpfulidiot.com/a-wired-sonoff-zigbee-alternative

Again, basic info for experienced Tasmota and Z2T users here :

https://templates.blakadder.com/ewelink_ZB-GW03

and again there is also a discussion about it in Home Assistant community forum here:

https://community.home-assistant.io/t/zb-gw03-ewelink-ethernet-zigbee-gateway-now-hacked-with-tasmota-zbbridge-so-can-be-used-via-mqtt-or-as-a-remote-zigbee-adapter-with-home-assistant-zha/341223

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Category: Controller Related to interaction with other platforms Status: Needs Info Needs more info before action can be taken Type: Discussion Open ended discussion (compared to specific question) Type: Feature Request Add a completely new feature (e.g. controller/plugin)
Projects
None yet
Development

No branches or pull requests

5 participants