Dealing with "Derived" Observations

[justinschembri]

Hello all,

I'm seeking some guidance on how to deal with what I'm calling "derived" observations. A derived observation is an observation which can be calculated, derived or generated from, and is therefore a child of, a real observation. To illustrate, here are two concrete examples from separate applications I am working on:

• an accelerometer samples acceleration, the resulting waveform is integrated to produce an "equivalent" velocity and / or displacement which is derived from the real observation. Here the relationship between the acceleration Observation is one to one with the derived velocity or displacement Observation
• a thermal camera captures a raster image. From this raster image it is possible to derive and store Observations such as maximum temperature, minimum temperature, etc. Here each of these observations are children of the parent image, and the relationship is one to many.

How would you represent this in the STA model? I'm exploring these avenues:

• not represent at all? Consider this to not be a modelling problem , and let applications yield the derived observations on the fly as needed. This has a downside if the derived observations are compuationally expensive.
• extend the model with a DerivedObservation class? A DerivedObservation is a subclass of an Observation.
• extend the model with a DerivedDatastream entity? Similar to the above, except passing over the problem to the datastream.

I'm wondering if anyone has experience with something like this; all opinions welcome!

[hylkevds]

This discussion would also fit nicely on the official SensorThings GitHub discussion list.

There are many aspects to the question of "Derived" Observations, but in the end the question is always: what does one want to make available to the client.

To start off, all Observations are derived. Electronic sensors generally measure a voltage, that gets translated into a value in a different unit. And even when I read a mercury (or oil) thermometer, I actually measure the position of a meniscus, that is translated into a temperature. These original measurements are not available to the end user, and the derivation details go in the Sensor or Procedure description. This is most likely also the case for your accelerometer example. The high frequency acceleration measurements are probably not available in the STA service, so you can just state in your Sensor metadata that it derives its measurements from acceleration data.

In the case of aggregations, like your raster example, the Sensor for those aggregations is the algorithm that does the aggregation. This is also the reason that in the update from O&M v2 to OMS, the class Sensor was renamed to Observer. A mathematical model or aggregation algorithm can also be the Observer (or in STA the Sensor). When the aggregation happens in time, this should be made clear by making the phenomenonTime of the Observation a time interval. When the aggregation happens in space, this should be made obvious by making the Feature an area, and not a point. If the original data is not available to the user, then the Sensor/Procedure can describe the combination of the original sensor and the aggregation done on the data.

In the case that the original data is available, but not in the STA service, the Datastream or the Observation can link to the original data. In STA v2 the Relations extension will the method of choice for this link.
If the original data is also in the STA service, then the Datastream can point to the original source Datastream(s). For example, this MultiDatastream holds the daily aggregate data for this Datastream. Note that in that service the "original" data itself is an hourly aggregate, as made clear by the phenomenonTime being hour-intervals, though since we didn't have any data for most of the Sensors, I didn't bother making a separate Sensor for the aggregation algorithm and just link to the original Sensor.

Subclassing of STA classes is never needed. Soft-typing is much easier to deal with for both client and server.

[justinschembri]

Hey Hylke, thanks for the reply which I've digested. First things first, and this is a question for @fterpstra - I guess we might end up having some discussions which fit in the official STA github repo. Should we have them here, or there and link them here?

Some observations (pun intended) from my end:

... but in the end the question is always: what does one want to make available to the client.

I've found that what needs to be made available to the client evolves. This is to say, I see situations when I'm persisting data and deriving aggregations as needs develop.

Your answer and the discussion at the last public meeting clarify that the Sensor entity may be real as well as virtual. Indeed it appears that in STA v2 the description is quite clear:

A Sensor is an entity that observes a property or phenomenon with the goal of producing an estimate of the value of the property. A Sensor may represent a piece of hardware, but a Sensor may also be a human or an algorithm implemented in software.

Thanks!

[fterpstra]

@justinschembri please have relevent discussions in the offcial STA repo and link them here. Either form discussions, issues or your section of the report.