Support for DDF bundles

[manup]

Important: This larger PR is still work in progress. The PR is meant as tech preview to try out bundles and improve the possible rough edges asap. While the technical side in this PR is almost done, the user facing UI needs to be figured out still.

Features:

• Load DDFs from raw JSON files like it has been since DDF introduction.
• Load DDF bundles according to a DDF policy.
• Support multiple versions of a DDF bundle, e.g. to try out new beta of a DDF and revert back to former version if desired.
• Add / update bundles at runtime to a setup via DDF REST-API.
• Device support becomes independent from deCONZ releases.
• Install new bundles without restarting deCONZ, aka hot-reload.

DDF Bundles

A bundle is just a single file with .ddf extension which contains the raw DDF JSON and all JS and JSON files it references. E.g. the files in generic/items and generic/subdevices etc.

Unbreakable

Each bundle has a unique SHA-256 hash over its content. The hash is signed by well known public keys, currently one for stable and one for beta bundles. These signatures are also part of the bundle.

And that makes bundles "unbreakable", for example changing an item in generic/items won't accidentally alter a stable bundle. But create a new one, which is only signed as beta — with the former stable bundle still existing.

• No more broken devices after updates (a difficult problem all Zigbee projects currently suffer).
• Setups can be kept running on a "never change a running system" per device base.
• The DDF validator already catches common errors and will be sharpened further on to ensure we get "good" bundles.

DDF load policy

Each device has two new top level API facing items attr/ddf_policy and attr/ddf_hash. The policy specifies which bundles (or raw JSON DDFs) are loaded for a device. The attr/ddf_hash in combination with the pin policy can be used to pin a specific bundle to a device.

Policy	Description
latest_prefer_stable	(default) Use latest DDF bundle: beta signed one if no stable available.
latest	Use latest DDF bundle either beta, stable signed or un-signed.
pin	Use and stay on bundle given by its <sha256-hash>.
raw_json	For development like before bundles existed just use raw .json DDF files.

To not cause havoc in the developer community, the raw JSON files, e.g. devices/ directory, will be still part of deCONZ and are usable as they are used to be.

In the same spirit there are two new directories, a deCONZ packaged bundles/ and a $USER location bundles/ from which .ddf files are loaded.

Performance

Loading bundles is ca. twice as fast as loading all raw JSON files. Further new in this PR only those DDFs are loaded into memory for devices which are actually present in the setup. The code was designed to crunch through thousands of DDFs easily since at some point we may end up with 3K to 5K bundles in total. We don't need to load thousands DDFs if a setup only has 30 distinct device types.