[REQUEST] Give feedback/input and consider adopting support for zigpy's upcoming open standard and open file format of "index.json" for Generic OTA providers?

[Hedda]

Please consider looking into this related sub-project concept to document a JSON schema as an open standard for "Generic OTA providers" Zigbee firmware OTA metadata that puddly is working on (for the zigpy project) whose formatting possibly has the potential to in the future become a universal open standard and common open file format that can be reused within the Zigbee open source scene in the future if it is adopted by others open source Zigbee framework/gateway projects as a shared common standard:

https://github.com/zigpy/zigpy/wiki/OTA-Information-for-Manufacturers

For reference also check out the related discussion in puddly's draft PR here for zigpy which will add support for Generic OTA providers:

zigpy/zigpy#1165

Implements the format specified in https://github.com/zigpy/zigpy/wiki/OTA-Information-for-Manufacturers:

zha:
  zigpy_config:
    ota:
      remote_providers:
        - url: "https://fw.zigbee.example.org/ota.json"
          manufacturer_ids: [0x1234, 0x5678]
        - url: "https://fw.zigbee.example.org/ota-beta.json"

This allows for new providers to be added to zigpy without extra code. This also allows for manufacturers to distribute test feed URLs to customers without deploying them globally.

By the way, puddly has also added a new feature to the "zigpy-cli" tool that manufacturers can use to produce a skeleton JSON index from a given set of OTA files, and that way make sure manufacturers OTA files are correctly-structured so that they will be compatible with zigpy out-of-the-box. See the README.md for the "zigpy-cli" tool:

https://github.com/zigpy/zigpy-cli/blob/dev/README.md#ota

This work by puddly is a follow-up to these previous zigpy discussions about standardizing of a generic format for index.json support:

zigpy/zigpy#535

zigpy/zigpy#654

Backstory:

zigpy is a suite of open-source libraries implementing a Zigbee stack that is used in several popular open-source home automation projects, including zigpy (Zigbee library used by Home Assistant ZHA integration, Zigbee Plugin for Jeedom, and Zigbee Plugin for Domoticz, and more).

puddly has previously also co-designed the "Open ZigBee Coordinator Backup Format" which is another universal open standard and open file format that has been adopted by he above mentioned projects that depend on zigpy as well as a few other projects are do not rely on zigpy (including zigbee-herdsman which is the Zigbee framework library used by Zigbee2MQTT and IoBroker )-> https://github.com/zigpy/open-coordinator-backup

While I do not know his motivation for sure I can make an educated guess as I know that puddly now works for Nabu Casa together with a few other lead developers of Home Assistant, with him specifically very recently becoming employed there to work on the underlying zigpy framework libraries which Home Assistant's ZHA integration depends on, and I assume that it should now be their interest to try in the longterm try to improve standardization of some decentralized things in order to offer a better experience for end users, especially with such things like Zigbee firmware updates via OTA (Over-The-Air) being one of the major things that Zigbee device manufacturers are not collaborating on.

It is also absolutely true that Koenkk already published a index.json in his https://github.com/Koenkk/zigbee-OTA repository that has relatively speaking already been widely adopted within more than a few open-source DIY home automation projects, however, if puddly's scope for the upcoming standard that will be used by zigpy is broader in order to try to cover more possible use cases I think it would be a very good idea for you and others to look into this and if possible try to get in on collaborating with your input early on in this effort on the documentation of it by at least giving your feedback on the concept and formatting, as chances are high that at least a few companies will sooner or later adopt their specific formatting of a universal open standard and common open file format.

Therefore, I believe that from their point-of-view as well as other projects willing to maybe adopt it in the future this idea of having a documented universal open standard and common open file format that can be used universally even by completing companies so they can simply point at as this documentation as a default reference which should directly be beneficial for all Zigbee projects that adopt that universal/common open standard as well as any companies that start to use this, and because Nabu Casa / Home Assistant and other projects that adopted this will have this documented universal/common open standard in one shared public location you can then not only share that standard publicly with multiple device hardware manufacturers but also try to convince all and any companies that they partner with to also adopt this universal/common open standard as then their devices can get out-of-the-box OTA support from many different projected without any further customizations being required.

For reference, Home Assistant / Nabu Casa recently started a partnership program which could indirectly allow them to put some pressure on those companies they partner with to adopt this new open standard if hardware manufacturers want to earn the right to use "Works with Home Assistant" badges for their devices once they tested compatibility. See:

https://www.home-assistant.io/blog/2022/07/12/partner-program/

https://partner.home-assistant.io

https://www.home-assistant.io/blog/2022/10/13/third-reality-partner/

https://www.home-assistant.io/blog/2022/07/27/leviton-partner/

[pipiche38]

It is not the aim of this project to produce any firmware, so I do not really understand the purpose of this .
The only benefit would be if zigpy provides an open source library to access such file and the corresponding firmware, but right now it is based on a zigpy-cli which is CLI by itself

[Hedda]

No I understand the goal/aim of that specific subproject is to try to convince many Zigbee hardware manufacturers to start offering their OTA provider servers in a uniform and standard manner with the same JSON schema formatting so that the zigpy library does not need custom download code for different manufacturers and instead the end-user will simply be able to add new URLs without having to touch any code. That is, it is trying to solve today's dilemma of zigpy library having to have custom download code for each OTA provider which means that end-users can not just add additional OTA providers by simply proving a new URL that the manufacturers provide:

https://github.com/zigpy/zigpy/blob/dev/zigpy/ota/OTA_URLs.md

https://github.com/zigpy/zigpy/blob/dev/zigpy/ota/provider.py

The benefit would be having a common open standard that any manufacturer can use that you only need to write one parser for.