Enrichment: Support for a wildcard/placeholder identifying the changed feature in order to enrich e.g. its definition

[w4tsn]

I've stumbled upon the following use-case with the enrichment feature introduced in 1.1.0. I'm opening a WS connection and receive change events of many different things. The events are from different features which I don't know at the time of opening the WS connection.

Together with the state change events I also need to know the feature definition field. Right now I take the feature of an incoming message and query it's definition for each event. What I'd like to do is to request the corresponding definition of the feature the updated property belongs to.

[thjaeckle]

Sounds like a good idea/enhancement.

[BobClaerhout]

We are using the enrichment feature as well and encountered the same use-case so would be nice to have this addition. I'm a bit confused however with the title of this issue. How would wildcards help you in this? From my point of view it would be nice to enrich a message with properties relative to the feature. If you do that, you can enrich a message with the definition but you can also enrich it with any metadata (assuming the metadata is the same on all features of course)

[thjaeckle]

Well, using e.g. extraFields=features/*/definition would allow you to get all Feature definitions of a Thing enriched when it is updated, so e.g. as Ditto Protocol event:

{
  "topic": "org.eclipse.ditto/fancy-thing/things/twin/events/modified",
  "headers": {},
  "path": "/features/temperature/properties/value",
  "value": 23.42,
  "revision": 34,
  "extra": {
    "features": {
      "temperature": {
        "definition": [
          "org.eclipse.vorto:Temperature:1.0.0"
        ]
      },
      "humidity": {
        "definition": [
          "org.eclipse.vorto:Humidity:1.0.0"
        ]
      }
    }
  }
}

And as you have in the "path", that the "temperature" feature was changed, you can find out its "definition" as well.
I agree that this could be optimized (in order to only send the "definition" of the actually modified feature), but as a first step this is much better than doing another roundtrip for fetching the definition, don't you agree @BobClaerhout ?

[BobClaerhout]

yes, sure. as a first step this is excellent! But indeed, my question/remark was more concerned the fact that you would receive the definition of all features. But again: excellent first step!

[thjaeckle]

I don't have a good idea (yet) to specify somehow in the existing syntax to only retrieve the definition of changed features as extraFields.
Not sure if this is even the right "place" to define that this information should always be included.

Any ideas from our community?

[thjaeckle]

A simpler alternative than providing the wildcard in the enrichment would be the proposal in #874
Kudos to @jokraehe for that idea.

[w4tsn]

As for the syntax you could potentially introduce a new operator like + which then has the semantics of only returning properties as extra fields at this level matching exactly what is sent in the signal. Opposed to the * operator which returns all the properties / fields of a level.

So if I specify extraFields=features/+/definition the enrichment should look at the signal path and use the same level as a kind of replacement. Giving me:

{
  "topic": "org.eclipse.ditto/fancy-thing/things/twin/events/modified",
  "headers": {},
  "path": "/features/temperature/properties/value",
  "value": 23.42,
  "revision": 34,
  "extra": {
    "features": {
      "temperature": {
        "definition": [
          "org.eclipse.vorto:Temperature:1.0.0"
        ]
      }
    }
  }
}

Regardless of this I think #874 is a good approach.

[desislava-marinova]

@thjaeckle, is it possible at the moment to get all definitions using extraFields=features/*/definition?

IMHO, extra fields with wildcards should be evaluated towards signal path or value (in case of several features/properties update).
I could not imagine a case where the user will need the definitions of all features of a thing while a single feature or a subset of features are changed, the definition (or any other feature properties) could be useful in the context of the changed feature, isn't it so?

[thjaeckle]

@desislava-marinova no, currently using wildcards is not possible, that's what this issue is about.
However #874 would provide a much simpler approach to always include the feature "definition" for feature modification events.

Using wildcard in the extraFields might still be useful in order to e.g. enrich all temperature values of several sensors, e.g.:
extraFields=features/temperature*/properties/value
But IMO this looses urgency when we would have #874

[desislava-marinova]

@thjaeckle, thanks.
I fully agree that wildcards in extra fields are very useful no doubt, I have commented only the features/*/definition case.

[ctron]

I would also be interested in this feature.