Allow add-ons to provide custom meta-data on items

[kaikreuzer]

This is a continuation of the discussion at #1093 (comment).

We have multiple add-ons (HomeKit, Alexa, ImperiHome, ...) that want to expose items to external systems and which require some special meta-information like a mapping to the type system of the remote system.

So far the approach was to use semantic tagging, but this has the drawback that it isn't possible to have different meta-data for different add-ons and to decide which ones to include or exclude on a per add-on basis.

I think we therefore need some other mechanism to provide that information. So far my recommendation usually was to come up with an add-on specific configuration file and create a dedicated REST endpoint for exposing the data.
@digitaldan challenged that and fears that we will end up with too many endpoints that would fragment the REST API and be a potential security risk.

Also, we should have a common way how to configure such meta-data - we should not force every add-on to come up with its own UI to do this job.

[martinvw]

A very generic idea. So what if add-ons could implement a certain class to provide scoped (prefixed like Alexa:) meta-tags. The ui could then provide a drop down based on all those provided tags

[anthonygillet]

Continuing the discussion from 1093...

Note that a boolean flag (expose/don't expose) won't be enough in many situations - you might want to add a specific homekit-type mapping to one item and a custom label to use with Alexa on another item - so I am afraid, the UI will have to be a bit more complex...

Could you elaborate on what you mean here? I'm not sure exactly why a boolean flag would not work. I figured that the complexity should be pushed down to the individual addons. Once an item is exposed to an addon, the addons themselves could provide additional metadata if necessary. For example, HomeKit internally will need to map the OH semantic tags to HK accessory types. Shouldn't it be up to the HomeKit addon to provide a way to configure that if desired, rather than OH?

You mention custom labels - I presume you mean the ability to specify alternative names for items to be controlled by voice. That seems like a pretty complex issue. For now, users can achieve multiple labels / alternative voice names by simply having multiple items that work together through groups, rules, or sharing channels. (not ideal, I know)

I agree that you don't want to get into a situation where you have to live with a temporary solution forever. However, a simple boolean UI would unblock development of this (seemingly highly-desired) feature, and keeping it simple would allow iteration later if more control is desired at the OH level.

[digitaldan]

I'll start with what I have done so far with a simple Alexa binding, maybe we can talk about what works and what does not with this approach. Note that I am very neutral on this solution, so I'm not trying to push this over something else, its just what I came up with a after a few hours of prototyping.

By default (today) Alexa will use any item that has voice tags exposed. For simple use cases this just works.

I also have a binding that allows you to select which items will be exposed to Alexa. This still requires tags to associate the item -> voice type, but you can decide which tagged items will be exposed.

Benefits:

1. Backwards compatible with what we have today
2. Somewhat simple (if you consider voice tags simple)
3. Uses the ESH configuration framework, so the paper ui already supplies a UI for item selection.
4. Uses the ESH configuration framework, so we already have a REST endpoint to retrieve the items

Cons

1. Not very granular, you can't map Alexa items to one voice type, and HomeKit to another
2. Still uses tags, and we don't have a UI tag editor yet (tags seems to be the first challenge users have)
3. Adding more features to configure may require extending the ESH config framework, or writing a custom GUI/UI (which I would like to avoid very much)

Here is what the Alexa configuration looks like when pulled from the REST interface:

{
  "exposed-items": [
    "Light_Great_Lights",
    "Group_Kitchen",
    "Group_HVAC_Upstairs",
    "Group_Second_Floor",
    "Group_First_Floor",
    "Group_HVAC_Downstairs",
    "PoolVoiceControl",
    "PoolWaterfall",
    "Light_Outside_Front_Fountain",
    "Group_Back_Yard",
    "Group_Front_Yard",
    "Lock_Front_Gate_Lock",
    "Light_Outside_Aux",
    "GarageDoorContact",
    "Sonos_sync_all",
    "Group_Casita",
    "Group_Dining",
    "Group_Garage",
    "GarageDoorSwitch",
    "Group_Great",
    "Group_Guest_Bed",
    "Group_Living",
    "Group_Master_Bed",
    "Group_Outside",
  ],
  "expose-all": "false"
}

and since a picture is worth a thousand words, here is what the paperUI looks like:

[martinvw]

I must say that I also liked the imho intuitive way I used homebridge with OH1.8.

There I just had a sitemap to expose items to homekit, the sitemap allowed:

• Overriding of the labels
• Selection which items should be exposed
• More or less mapping of types although that was not fine grained (enough).

Could such a solution help us? Or is this too much of a work around.

[digitaldan]

There I just had a sitemap to expose items to homekit, the sitemap allowed:

I have never been a fan of overloading the current sitemap structure for this, but I understand your point about simplicity, and I think its worth talking about.

One aspect to this is that we are still dealing with a UI, just a voice UI vs a visual UI. So, I guess one thought is do we have something similar to the sitemap (voicemap?) where you can lay out what voice items, types and labels you want. This would also very much help with things like Thermostats, where we need to group items together. You could then have a voicemap for each type of service (Alexa, homekit, ....) . Or maybe we extend the sitemap to be more voice friendly?

My hesitation with using sitemaps as is is that we are overloading one concept to fit another (frames, images, etc... do not fit voice). My other is that sitemaps are a OH1 concept and there is no built in storage for them other then flat files (aka no JSON storage).

[digitaldan]

Could you elaborate on what you mean here? I'm not sure exactly why a boolean flag would not work. I figured that the complexity should be pushed down to the individual addons. Once an item is exposed to an addon, the addons themselves could provide additional metadata if necessary. For example, HomeKit internally will need to map the OH semantic tags to HK accessory types. Shouldn't it be up to the HomeKit addon to provide a way to configure that if desired, rather than OH?

The Alexa binding i have prototyped I think accomplishes this, and to your next point, if you need to have items exposed differently, you can create duplicate items with different labels and tags and only expose those to a particular service.

This works, I think, but I'm not sure is ideal. There are now several places a user has to change things, and it can be difficult to explain. Tagging an item by itself has been difficult for users, but trying to explain now how to duplicate items, tag them differently, then go to a particular binding and choose those items to expose is a lot of work. Of course, thats exactly the path I went down :-)

[martinvw]

One aspect to this is that we are still dealing with a UI, just a voice UI vs a visual UI. So, I guess one thought is do we have something similar to the sitemap (voicemap?) where you can lay out what voice items, types and labels you want. This would also very much help with things like Thermostats, where we need to group items together. You could then have a voicemap for each type of service (Alexa, homekit, ....) . Or maybe we extend the sitemap to be more voice friendly?

What about this, maybe introducing types in/of sitemaps? @kaikreuzer wdyt?

[kaikreuzer]

Could you elaborate on what you mean here? I'm not sure exactly why a boolean flag would not work. I figured that the complexity should be pushed down to the individual addons. Once an item is exposed to an addon, the addons themselves could provide additional metadata if necessary.

@anthonygillet: Well yes, this was exactly my point: We need more meta-data than just a boolean flag. And if we have a mechanism for add-ons to provide that meta-data, they can also provide the "exposed or not" information, such that there is no need for having that information provided by the core framework.

For now, users can achieve multiple labels / alternative voice names by simply having multiple items

Yes, but this is really ugly indeed. If we have a mechanism for meta-data, a custom label would imho fit in there without any problem as well and be a much nicer solution.

@digitaldan: Your prototype looks nice and it is what can be done with the current possibilities.
What would be needed on top is not only to "select" single items in the configuration list, but to be able to provide a more complex configuration for them. So instead of having a "multiple=true String" configuration parameter, we would need a "Map<String,AlexaConfiguration>" parameter and on top of that it should be possible to provide a config-description for AlexaConfiguration. The UI could then offer the list of items and for every selected one have an "edit" field, which could open another config dialog. Alternatively, the UI could work with a two column layout to allow editing the configuration. @SJKA & @htreu: I know this would be yet another level of complexity for config-descriptions, but from different discussions I feal that we sooner or later will have to think about the support of Maps here, wdyt?

maybe introducing types in/of sitemaps?

@martinvw: As @digitaldan mentioned, the current sitemaps are not state-of-the-art for ESH and rather due to be replaced by something more UI-editing-friendly. So from a technical standpoint alone, I would not suggest to go that way. Furthermore, I would not reduce the discussion here to "voice", but we are actually talking about any kind of add-on that wants to expose stuff in some specific way. The required meta-data can also be very different for each add-on, so trying to make sitemaps suitable for all, probably won't work.

I'd think the sketched solution from above that builds on @digitaldan's prototype should be a possible way to solve the requirement. Note that from a UX perspective, it is pretty different to using custom tags for such meta-data:

1. Custom tags: The user would browse through the items and add meta-information for Alexa, Siri, whatever to each of them. The "leading" entity is the item here.
2. Add-on configuration: The user would go into the configuration of the "Alexa add-on" and then specify the meta-information required for Alexa per item (or maybe even globally for all items or groups). Afterwards he would go into the Siri configuration etc. The "leading" entity would be the add-on.

I cannot say which approach I prefer, but I have a tendency towards option 2 as it keeps the modularity aspect visible: If an add-on isn't installed at all, there also is no way to configure it. On the other hand, only option 1 so far has a solution for providing the meta-data through the REST API; so we might actually have to come up with some mix of both approaches. Maybe we can declare it as a new add-on configuration concept and NOT introduce a generic Map support in configurations for the start, but instead have the framework support this special requirement by merging such configurations with the items on the REST API. The problem would then only be that we also need a dedicated UI for editing such information... All not too easy, I am afraid :-(

[sjsf]

I feal that we sooner or later will have to think about the support of Maps here, wdyt?

We might. But this will introduce an awful lot of complexity, both in terms of UI design as well as on keeping the information consistent and staying generic at the same time in the data model. I therefore would like to avoid unless really needed. So we should cautiously check if the use-case we have in mind here really makes sense like that or if the modeling really is correct from a conceptual point of view.

From reading the above discussion, it looks to me like the additional configuration indeed is a property of the item and not of the add-on, while still being specific to the add-on. So why not combine the two suggested approaches like following - it more or less follows the path that you indicated in your last paragraph:

We could treat such configuration as a "link" between an item and an add-on. It would contain the item name and the add-on name as "primary keys" and would carry a specific configuration . The add-on provides a config description and therefore defines what can be configured in the realm of this add-on. UIs then may decide themselves whether they want to start from the item (selecting the add-on) or from the add-on (selecting the item) for this specific configuration part. As we have config descriptions, this should not be too complicated to embed in the existing UIs using the available mechanism for generically rendering configs. The available "add-on-links" could then be rendered into the /rest/items resources (maybe only upon request by an attribute) so that clients may use them.

[htreu]

I second @SJKA´s approach.
Bundles may provide an implementation of an ItemMetaDataConfigDescriptionProvider or specific config descriptions via xml. The configuration may be stored relative to the item with a binding or service id identifying the corresponding ItemMetaDataConfigDescriptionProvider.
The REST rendering might be done in place or with HATEOAS links per binding/service id.

[kaikreuzer]

I also like the idea, but it will also mean that we introduce yet another "entity" that needs to be handled by the infrastructure.

FTR: I have started prototyping a proof of concept - the rough architecture is the following:

• A new MetaData class, which has itemname+extensionid as a key and a Map as value.
• Introduction of a MetaDataRegistry with the common ESH design pattern (i.e. with MetaDataProviders, a ManagedMetaDataProvider, a /metadata REST endpoint, Added/Changed/Removed MetaData events)
• Additionally "mixing in" the meta data on the /item REST endpoint through query parameters (so that e.g. items with Homekit meta-data can be requested in a single operation).
• For item DSL support, I would suggest to take over the "BindingConfigProviders" - after we introduced the "channel" binding configs that deal with the links, there is not much use of that syntax besides the channels anymore. We could therefore say that we can simply add homekit="...." to items and pass that information directly to a GenericMetaDataProvider.

A few thoughts/issues I came across:

• We will need some mechanism to determine when to automatically delete metadata entries from the managed provider and whether it should be deleted if one part of the key (item or extension) is gone or only if both are gone.
• We cannot use Configuration as a value for MetaData, since we cannot depend from core on config bundle - we therefore have to stick to Map.
• The same question as for the other entities arises: How do we deal with edit requests for data that comes from other than the managed provider? E.g. the HomeKit addon might carry a rough HomeKit mapping which is provided through some read-only MetaDataProvider, while the user wants to add details through the Paper UI. How to handle it?
• We might use the same infrastructure for handling the ESH tags. This would solve the current issue that it is not possible to add tags through the UI to items defined in files.

[andylintner]

@kaikreuzer - in that prototype you describe, how do you see users managing the actual mapping to an add-on's type. It seems like a good way to solve "Should I expose this?" but not necessarily "What is this?". Currently, we do that with a tag like "Lighting". There's a proposal out there to use Channel Categories instead, with tags as a fallback. Do you envision this would instead become part of the metadata? It seems we then lose some of the auto-configuration we'd have from Channel Categories.

[marziman]

@kaikreuzer & @digitaldan:
Reading this discussion I am a little confused. I tought there is the PR of @SJKA refarding metadata. We could use that approach to sove this.
I think that the approach which @digitaldan mentioned and we used for the cloud connector is pretty cool.
As much I like the per item metadata, maybe dividing it per Addon is the better way in regards to UX.
The most important thing: How easy and intuitive can this process be for non nerd OH/ESH users.
I wouldnt mix the sitemaps into it, since solutions on top of ESH might not use themat all due to their fatness.

I think @digitaldan approach is a big step and we should keep in mind that we need to make this run, even without OH dependencies. Best is to engage the REST API and let the solutions decide how to show it in UIs.

Only some small ideas and inputs.

Still thought that metadata infrastructure is already in place..or lets say we have a PR :-)

BR Mehmet

[kaikreuzer]

Still thought that metadata infrastructure is already in place..or lets say we have a PR :-)

Surely there is: #4390 - it has already received quite some thumbs-up, and no down, so I would suggest that you look at that and provide your feedback where you might see problems.

@beowulfe Sorry, I actually missed your comment here on the issue, but I hope that the PR that I created shortly after answered at least part of your questions. So yes, the tags should imho become such meta-data and I think we will need both, categories and tags as a combination to get a full picture of the item (the category more for describing the type and capabilities, while the tags rather describe the purpose and the semantics).