Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mapping of json keys #34

Closed
simeonthefirst opened this issue Jun 30, 2023 · 6 comments
Closed

Mapping of json keys #34

simeonthefirst opened this issue Jun 30, 2023 · 6 comments
Labels
enhancement New feature or request

Comments

@simeonthefirst
Copy link

I have a usecase, where there are many different measurements and more may also be added over time. To help that i don't want to have a mapping for each measurement, but only one dynamic mapping that covers that.

Threfore i have to map the keys of the source message (eg. /devicename/measurementname { "series-name": 12 } ) to the mqtt message format of cumulocity where the measurement name and series names are keys not values . In this example from a topic to a json key ( [ _TOPIC_LEVEL_[1] -> $keys()[3] ] ) and from a json key to a json key for the series name: [ $keys()[0] -> $keys($eval($keys()[3]))[0] ] .
To put the value into the series i used: [ $eval($keys($)[0]) -> **.value[0] ]

I know that with that kind of mapping the order of the fragments is relevant what is not json like but i think thats the only option.

I have no experience with jsonata but my queries worked in the online jsonata tester (https://try.jsonata.org/) . The testing in the c8y app seems to have failed (screenshot).
image

Is it in somehow possible to map the keys or is this service limited to value mapping?

@switschel
Copy link
Member

Hm we can investigate that but full generic mappings are always very dangerous. A lot of bad stuff can happen when using it. Our core concept is that we use a specific topic as a key for a mapping. So when the topic is /devicename/measurementname the payload including keys should always be the same (except the values of course).
You can also use a template where you define all possible keys by hand and map them accordingly in just one mapping.

@sagIoTPower
Copy link
Collaborator

Hi @simeonthefirst ,

you can try to use the following sample, to generate dynamicaliy new nodes:

Screenshot 2023-07-03 at 18 08 35

This will transform the source msg. as follows:
Screenshot 2023-07-03 at 18 08 15

If your mapping requires more flexibility, then I recommend using the programmatic approach. This means writing your own mapping class in Java and upload this to the mapper. A sample implementation can be found here.
The extension mechanism is documented here.

Pls. let me know if this helps.

--Christof

@simeonthefirst
Copy link
Author

Hi,
thank you ! Your exampe, @sagIoTPower gives the flexibility to ignore the key-string on the device-side message what is part of what i wanted to achieve.
It does not enable me to change the measurement or series name on cumulocity side of the mapping. To realize a mapping like this (see below) , i will need to implement an extension. I will let you know when i do.

(device side)
{
 "Measurementname": "Airsensor",
 "Seriesename":"Humidity",
 "value":10,
 "unit":"%",
}
-->
(cumulocity side)
{
  "Airsensor": {
    "Humidity": {
      "unit": "%",
      "value": 10
    }
  }
 "time": ....
...
}

@sagIoTPower
Copy link
Collaborator

Hi @simeonthefirst,

I added a new setting RepaiStrategy = REMOVE_IF_NULL and added a sample, mapping number 23 ( flexMeasurement).

Could you pls. take a look if this helps and fits for your use case?

Regards Christof

@sagIoTPower sagIoTPower added the enhancement New feature or request label Jul 15, 2023
@simeonthefirst
Copy link
Author

Hi @sagIoTPower ,

thank you! I did not had the time to test it jet, but took a look at the example. For me it looks like the measurement-/fragmentname can now vary between different pre-known names. Or do i get it wrong?
More important for me would be to set the measurement-/fragmentname and/or also series name to something you don't now at the time when you create the mapping. I'm sorry if this was a misunderstanding.

best regards,
Simeon

@sagIoTPower
Copy link
Collaborator

Hi @simeonthefirst,
the graphical mapper aims to cover mapping where the structure of the messages is well understood and stable. This allows for a certain flexibility.
In your case you would like to define a mapping for messages with varying structure.
Could you take a look at the sample processor extension for JSON payload?
The sample defines a mapping as a processor extension in a Java class. This allows a higher degree of control over how to handle different payload structures.
The additional RepairStrategy.CREATE_IF_MISSING helps in this case as well.

The release v.3.1.7 covers this new strategy.

Regards

Christof

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants