Entity creation from UI

[farmio]

We'd like to be able to create HA KNX entities from the UI instead of YAML.

Let's see if we can come up with ideas on how this should look like and work.

[probert94]

A manual configuration using the UI would already make it a lot easier to use the integration, at least for users not confortable with YAML.
The first idea, that comes to my mind is to first select the entity type (Binary Sensor, Cover, Light etc.).
Depending on the selected type the next step shows more or less fields. For the binary sensor, for example, only a "state_address" is required, a light instead is way more complex and even depends on more details (dimmable, tunable white, rgb(w) etc.).
That way a user would not need to remember all of the property names.
The downside of this approach is, that an experienced user will probably be way slower then in YAML. Some people even create custom scripts that convert a group address export into a YAML.

So the UI should probably allow some kind of patch creation. Usually the structure in ETS follows some standards, so addresses of the same entity are either grouped in blocks or in several middle groups.
Example:
Light 1:

• switch: 1/0/0
• state: 1/1/0 or 1/0/1
• relative dimming: 1/2/0 or 1/0/2
• absolute dimming: 1/3/0 or 1/3/0
• dimming state: 1/4/0 or 1/0/4
• ...

Of course there are quite a few things to consider, for example TW / RGB(W) might be separated in multiple blocks (one for each color).
Also, in the "block" system one might not reserve the dimming address for non dimmable lights etc.
So adding this kind of logic will already be pretty complex.

Last but not least, the ideal solution would of course be an import of a KNX project with auto detection of the entities.
Unfortunately this is pretty complex and probably won't work in all situations / with all projects.
However, I found a pretty interesting (ggerman) video from PEAKnx, where they explain how the project should be structured in order to detect the entities:
https://www.youtube.com/watch?v=c-fKEkp9LBs

[farmio]

The possibility to script your config creation will probably not go away.

Also the UI created config will have to be stored somewhere, and this will probably be some json in .storage/knx as it is easy to work with and HA provides good tooling for that. So if someone liked to, this json file can be created / extended by custom scripts just like yaml can currently.

That said, I don't think we have to get rid of YAML (at least in the near future) so this will also be an option still.

[farmio]

Imposing naming or project structure conventions for our users seems a bit harsh to me.

The most common case for our users is that they already have a working installations when they start to use HA. Some users even don't own ETS licenses, so they wouldn't even be able to change their projects accordingly. Others may use different systems that already impose some structure on their own (like your example) that may be incompatible.

For those who are committed to a specific GA numbering schema, scripting could be a viable option though.

[probert94]

Imposing naming or project structure conventions for our users seems a bit harsh to me.

I would not force them to use an exact structure and namig, the manual configuration always needs to be possible.
But in order to provide a simple auto import you probably must make some assumptions about the project structure and naming.
As KNXA itself provides some guidelines regarding ETS projects and there are a few other popular structures teached by books like "ETS Schnellkurs" and so on. So (hopefully) most projects will follow some rules and at least some entities could be auto detected.

So my idea is to make the configuration simpler (as automatic as possible) for users with well-structured ETS Project while still providing a simple manual configuration for all others.

[marvin-w]

I honestly doubt that there is a unified way (as of now) of how we can achieve automatically configuring entities.

Instead I'd opt for an (improved) ETS Look&Feel where users can see their project in the UI and then we could potentially work with context menus to start creating entities based on this user input. This would work really well for entities that have one or two DPTs but not so much for our bigger ones like climate and light. For those we definetly need dedicated views.

In this views we can use the parsed project data to prefilter the available DPTs during selection to increase the user experience while still allowing the user to manually switch the filtering off (i.e. if the DPTs are not maintained accordingly in the project file).

[farmio]

Looking into peaKnx it seems they not only expect certain numbering schema, but also maintain a database of devices and their possible parameters to be able to parse correctly.
And from what I read it doesn't sound that reliable.

See (German) https://knx-user-forum.de/forum/supportforen/peaknx/1837925-import-geräte-parsing-buggy?p=1868544#post1868544

[probert94]

I did not know about that, I just did a quick research on how other visualizations deal with the project import and it looked pretty good at first glance.
However, it seems like parsing the project is even more complex that I thought.
So it will probably be better to focus on a easy manual entity creation instead of an automatic one.

[ztardik]

I was working with a visualisation SW that offers auto-import (Divus). They approach is to offer filters to the user. For example, the following GA's can be imported easily:

0/0/0 [Kitchen] Spot /c
0/0/1 [Kitchen] Spot /sta
0/0/2 [Kitchen] LED /c
0/0/3 [Kitchen] LED /sta
0/0/4 [Kitchen] LED /dimm

User set's the filter as - command: '/c', status: '/sta', dimming brighter/darker: '/dimm'
The GA description is used then to determine what belongs together as one entity.

In small projects the manual selection is ok, but in case of bigger ones I would like to see some kind of preselection. Example would be an apartment building I was working on some times ago:

0/0 Building A, Floor 0, Apt1, Lights,
0/1 Building A, Floor 0, Apt1, HVAC,
0/2 Building A, Floor 0, Apt2, Lights,
0/3 Building A, Floor 0, Apt2, HVAC,
0/4 Building A, Floor 1, Apt1, Lights,
0/5 Building A, Floor 1, Apt1, HVAC,
0/4 Building A, Floor 1, Apt2, Lights,
0/5 Building A, Floor 1, Apt2, HVAC

GA's from the example above. What I would like to see is the option to choose only one area to work with at any given moment, instead of scrolling trough all addresses or having to manually write each into a search box.
It would be great if when se selecting 0/0/2 [Kitchen] LED /c you got automatically shown 0/0/3 and 0/0/4 as related GAs.

[probert94]

@ztardik thats an interesting approach. So instead of needing a specific structure and naming-convention to auto-import all at once, you import one entity-type alone and give hints about the used names.
I could imagine something like this:

Basically you specify some kind of address range, which could be a main-group, a middle-group or maybe a range (f.e. 1/0/0 - 1/0/10).
In the example it is the main-group "1/" which is used for lighting.
This might be needed, when you have other GAs with similar naming. For example, covers have a state address too.
Then you specify a pattern for each type of GA needed for the entities. The example uses dimmable lights, so there are 4 patterns (switch, state, brightness and brightness state). Each pattern consists of a type (starts with, ends with, contains, regex or whatever makes sense) and the value string.
In the example GAs for the switching addresses end with Switch, GAs for the state address end with Switch State etc.
I also added a checkbox to specify if non connected GAs should be ignored. The reason for this is, that in my project, I reserved the GAs for dimming, even if the light is switchable only. But of course they should be imported as switchable, not dimmable.
Then there are the additional properties, like the "synchronize" property, which can't be read out of the project. So the values set here are defaults for all imported entities.

Of course this is just a quick idea and needs a lot more elaboration. For example it is not clear how to connect different GAs to each other. One possibility would be the name, so that Living Room Ceiling Switch and Living Room Ceiling Switch State would result in an entity Living Room Ceiling with the GAs for switch and switch state set to the respective values.

Also this approach could become quite complex for RGB(W) lights as there you could have 1 GA for the color or all GAs (switch, state, brightness, brightness state) for each color.

[ztardik]

This is an example how most of my projects look like (everybody do it differently). Sometimes the main groups is the floor separator, sometimes the middle group. It depends from the size of the project. But basically you always have something fixed describing what it is and where it is.
The approach you described is good, flexible enough for most types of control. RGB(W) may be more complex, but how many times you have to deal with it? I think it's better to focus on the most used types and later add others.

Because all projects are different, automatic grouping is hard to make, but with implementation of filters, some clever sorting could check for same text in description which is not in filters (for example [Room 01] but ignore /c, /sta, /dimm ), then do something with it.
Type of control must be determined from the group type in the project. In this example there is two/three types of "/c":

 0/0/1 [Entry Hall] Spot /c
 0/2/1 [Room 01] SPV /c
 0/2/7 [Room 01] Cooling Output /c

In this case 0/0/1 is clearly a 1.001 switch, 0/2/1 is a 9.001 Temperature, and 0/2/7 can be 1.001 switch or 5.001 percentage.

[farmio]

For the more complex platforms, namely

• climate
• cover
• light

I think we could look into ETS Functions (and DPT lookup) for GA suggestions.

Although they don't provide all information we'd need for an entity to be created, we could speed up the config process with these.

[probert94]

I just had a quick look at another KNX tool, the Gira Project Assistant. It seems like it does not auto create entities but instead just import the GAs and let you cinnect them to entities.
This would of course be a great first (or second) step.
Wirh some additional logic it should then be possible to give better suggestions to which GAs are suitable for other addresses of the same entity (similar name etc.)

[probert94]

I created a quick Figma prototype for the entity configuration in the UI:
https://www.figma.com/proto/0XQAVAZY0kFSYOvS61XnXp/KNX?type=design&node-id=1-165&scaling=min-zoom&page-id=0%3A1&starting-point-node-id=1%3A165
In this prototyp the configuration is done in the configuraion dialog from the integrations panel, similar to what other integrations do.
However, I had a quick discussion about it with @farmio and we are not sure if the entity-configuration is not too complex for those dialogs. Especially when it comes to RGB(W) lights with different addresses per color it will get quite complex (4 addresses per color + the entity configuration).
However, I think the principle of the configuration could stay more or less the same no matter where it is located.
The basic idea is, that the addresses are configured using a combobox. This allows you to give suggestions for addresses (based on DTP, names etc.), search for addresses (using address or name) and enter custom values if needed.
I personally would avoid a drag and drop configuration like in the ETS as it seems pretty slow to me. Generally using the mouse is way slower then the keyboard.

[farmio]

SelectSelector style drop-down boxes with custom value option (and multi-select option, as most of our GA-config-keys accept multiple addresses) are used for the interface device trigger in automations too - I think it's a pretty nice way to configure GAs.

They also inherently support internal GAs (but these aren't available in the suggestions).

It's also possible to use that on mobile, whereas drag&drop would be a challenge.

[philippwaller]

@probert94 & @farmio, fantastic concept! Clustering entities into devices in Home Assistant seems like a solid strategy. The main challenge, though, is incorporating device-specific functions into the device object, like automatic shading for certain roller shutter actuators. Ideally, the device object should be versatile enough to include not only main functions but also link to device-specific entities. If possible, there should also be a generic device option (possibly at the HA Core level?), to cover specific use cases.

Currently, my YAML configuration is set up as if KNX devices were already integrated into Home Assistant. Each file represents a distinct device within Home Assistant. The schema works great and helps to keep consistency. However, managing all this through the UI would indeed be taking it to the next level.

Great work! Thank you!

[probert94]

I recently added another light source which I inserted in the middle of my lighting group addresses. So I had to move half of the addresses by one.
As the GAs are used as some kind of ID in Home Assistant, all the entity configuration (Name, Area etc.) as well as the history was wrong after that. The easiest fix I found was to delete all entities and recreate them.
So maybe a possibility to update the GAs of existing entities should be considered as well.

[farmio]

This can be done when we have UI config. We can simply use a UUID or an incremental counter to use for unique_id instead of having to rely on user input.

But then still your GAs will not auto-update. If you change GA numbers for whatever reason, you'll probably have to do that in the UI config too.

[heidle78]

Short question, in my KNX folder I can´t see the ETS project information - did I miss something? Monitor works great. Im imported my ETS project file and I can see all the information from KNX.

Beside that, good job!!!

[farmio]

Hi 👋! It's not released yet. We are still working on it.

[heidle78]

Thanks for your quick reply, was not sure if I made a mistake as the screenshots looked really promising and real. Can´t wait to get the new release with that feature - hope that simplifies the entity creation a little bit.

[farmio]

Currently it just shows a list of group addresses. We wanted to add some kind of "creation feature" before releasing. Still a lot to do though.

[Julius2342]

Probably a longshot here:

im still thinking if we one day might configure KNX devices via XKNX - without the use of ETS. My idea is to have a more-or-less standard configuration binary blob, optimized for HASS use. During the automated configuration process just the KNX addresses are replaced within the blob.

The System then knows which KNX addresses were assigned. this data then may be used for configuration.

:-)

[farmio]

Longshot for sure 😂

What devices would you think are good candidates for that?

Do you know if it was legally okay to ship a pre-defined KNX device application blob with xknx (or some sub-project)?

[Julius2342]

I would say any super-simple device, like e.g. https://katalog.gira.de/de_DE/datenblatt.html?id=658305 Program it, record the configuration. Change one single parameter e.g. von 1/23/42 to 1/42/23 and record the difference.

Regarding shipping: I would leave it up to the user to download the knxprod-zip files.

[ztardik]

I don't this this is possible or even worth spending time trying. You are saying it's a simple device, but look at the configuration options:

Each input (of two) has six different functions, and each function has different parameters. How many 'blobs' you have to create so that this become usable?
Many other devices are even more complicated. Of course one can make some basic setups available, but the usability would stay very low/basic.
What I think is worth trying is to make an alternative to ETS. A software compatible with ETS in any way possible, but with better/different workflow and usability.

[Damageplan1980]

Hi there,

although I don't use the Hager Domova TJA470 anymore, I always liked the way of adding devices to it.
There's a simple GUI where you can choose the type of device you want to add.
I give you a small example with a light:
You first have to choose the features your light supports.
There's a basic config where you can just set the group addresses for switching and the status.
You also can choose whether the light has color, is dimmable and whether it has color temperatur settings.
The KNX-Konfiguration section changes accordingly to the settings you choose.
At the input fields for entering the group addresses you can see what DPT is needed for every function.
If you click on the magniefier for searching your imported KNX project (group addresses) a popup opens where all the possible devices are filtered for the needed DPT.

Many greetings,
Matthias

Here are some screenshot: