Handling values with same timestamp in Data Lake.

[obermeier]

I have some Data with the same timestamp which I want to store in the Data Lake.

Something like this:

{"timestamp": 1685606975, "name": "sensor1", "value": 28}
{"timestamp": 1685606975, "name": "sensor2", "value": 30}

This leads to the problem that data in the Data Lake are replaced if they have the same timestamp.

What do you think about the idea to add the option to select a field as tag in the Data Lake config menu?
Is this to Influx specific?

This especially occurs if Json->Array is used as format in the adapter configuraton and a timestamp field is added.

[tenthe]

Hi @obermeier,
did you model the name as a dimension property when you defined the event schema?
If you do that, the data sink will automatically create a tag in Influx and you can have multiple events with the same timestamp.
To see the individual events, you can group the data in the Data Explorer.

[obermeier]

Hi @tenthe ,

thank you for the hint!
Very nice feature :)

[obermeier]

Hi @obermeier, did you model the name as a dimension property when you defined the event schema? If you do that, the data sink will automatically create a tag in Influx and you can have multiple events with the same timestamp. To see the individual events, you can group the data in the Data Explorer.

If I understand correctly, the Schema Editor of sources provides the option to set the property scope. From a UX perspective, would it be beneficial to include this feature to the JavaScript Eval processor as well?

[tenthe]

Hello @obermeier,
thank you very much for raising this issue. I think it's a valid point that we should discuss in more detail.
That's why I started a new discussion (#1654).
If you have any ideas you are welcome to write them there.