Issue by getting Observation from MQTT broker

[ghost]

Hi,
I posted a DataStream and several Observations via Rest to the Frost-Server.

Then i tried to get these Observations via MQTT broker.

The received JSON is as follows:
{ "phenomenonTime" : "2016-03-02T10:50:00.000Z", "resultTime" : "2016-03-02T10:50:00.000Z", "result" : "7.2", "Datastream@iot.navigationLink" : "http://localhost:8080/FROST-Server/v1.0/Observations('1f067586-604e-11e8-b7bf-c3302333d15c')/Datastream", "Datastream" : { "unitOfMeasurement" : { "name" : null, "symbol" : null, "definition" : null }, "@iot.id" : "t1edu.teco.wang/IBADENWR82/Datastreams/Air-Temperature-IBADENWR82" }, "FeatureOfInterest@iot.navigationLink" : "http://localhost:8080/FROST-Server/v1.0/Observations('1f067586-604e-11e8-b7bf-c3302333d15c')/FeatureOfInterest", "FeatureOfInterest" : { "@iot.id" : "t1edu.teco.wang/IBADENWR82/FoI/IBADENWR82/48.441/9.257" }, "@iot.id" : "1f067586-604e-11e8-b7bf-c3302333d15c", "@iot.selfLink" : "http://localhost:8080/FROST-Server/v1.0/Observations('1f067586-604e-11e8-b7bf-c3302333d15c')"},

which seems not correct:

1. unitOfMeasurement in DataStream entity is not returned correctly. I checked this via Rest, it's inideed saved in Frost-Server, but not sent via MQTT.
2. Maybe the entity FeatureOfInterest of an Observation should also be completely packed in JSON? Just like its 'DataStream'
3. Maybe the 'ids' and 'selfLinks' should not be returned? It makes little sense in the message.

Thanks a lot in advance!

[hylkevds]

What subscription did you use to get these Observations? The Datastream should not be expanded in the Observation, when getting it over MQTT. That would be a bug.

As to 3, the spec states:

Response:When a new entity is added to the entity set (e.g., a new Observation created) or an existing entity of the entity set is updated, the service returns a complete JSON representation of the newly created or updated entity.

So the question would then be what exactly "a complete JSON representation" is. We've interpreted it as everything, including self and navigation links, since you can always turn those off using the $select parameter.

[ghost]

I think it a BUG. I used 'paho.mqtt.client' to receive Observations. The received payload in msg is:

{
  "phenomenonTime" : "2016-03-01T22:50:00.000Z",
  "resultTime" : "2016-03-01T22:50:00.000Z",
  "result" : "1.8",
  "Datastream@iot.navigationLink" : "http://localhost:8080/FROST-Server/v1.0/Observations('191129a0-604e-11e8-b7bf-a3be7c3b0fe3')/Datastream",
  "Datastream" : {
    "unitOfMeasurement" : {
      "name" : null,
      "symbol" : null,
      "definition" : null
    },
    "@iot.id" : "t1edu.teco.wang/IBADENWR82/Datastreams/Air-Temperature-IBADENWR82"
  },
  "FeatureOfInterest@iot.navigationLink" : "http://localhost:8080/FROST-Server/v1.0/Observations('191129a0-604e-11e8-b7bf-a3be7c3b0fe3')/FeatureOfInterest",
  "FeatureOfInterest" : {
    "@iot.id" : "t1edu.teco.wang/IBADENWR82/FoI/IBADENWR82/48.441/9.257"
  },
  "@iot.id" : "191129a0-604e-11e8-b7bf-a3be7c3b0fe3",
  "@iot.selfLink" : "http://localhost:8080/FROST-Server/v1.0/Observations('191129a0-604e-11e8-b7bf-a3be7c3b0fe3')"
}

For sure I can get everything through REST. My point is that if it's worth it to put these links in the MQTT message instead of just expanding entities. When a user tries to get Observations over MQTT, his expectation is likely to be self-contained JSON messages with all interesting information, so that it can be processed right away. Otherwise he may have to query the information about FeatureOfInterst over REST or maintain a small cache locally to store these information. I think its the point of messaging over MQTT.

[hylkevds]

I agree it would be useful to be able to get the expanded datastream and feature of interest, but that would be a change to the standard, so that discussion should go on the Github page of the standard: https://github.com/opengeospatial/sensorthings

Can you also give us the JSON of Observation you use in those create requests? did they contain a datastream and FoI entity?
Did you create the Observation using HTTP, or using MQTT?
Did you create them using v1.0/Datastream(x)/Observations, or using v1.0/Observations?

[ghost]

I created the Observation via HTTP. Its DataStream and FoI are created separately. The @iot.id of the DataStream is specified in the URL and the @iot.id of the FoI is packed in JSON.

{"phenomenonTime": "2016-03-01T22:50:00.000Z", "FeatureOfInterest": {"@iot.id": "t1edu.teco.wang/IBADENWR82/FoI/IBADENWR82/48.441/9.257"}, "resultTime": "2016-03-01T22:50:00.000Z", "result": "1.8"}

[hylkevds]

It should be fixed in the latest master.
I did notice you've encoded the result value as a string in your Observation json, even though it looks like it is a number. Is that intentional? If you put it as a number in the json, then the server is able to treat it as a number when ordering and filtering.

[ghost]

Thank you for the quick fixing. You are right, now i convert the result to float before sending the Observation to the Frost-Server.
The Observation looks like:

{
    "phenomenonTime" : "2015-12-31T22:16:00.000Z",
    "resultTime" : "2015-12-31T22:16:00.000Z",
    "result" : 1.2,
    "Datastream@iot.navigationLink" : "http://localhost:8080/FROST-Server/v1.0/Observations('cbaff990-6280-11e8-b689-070f60f264ee')/Datastream",
    "FeatureOfInterest@iot.navigationLink" : "http://localhost:8080/FROST-Server/v1.0/Observations('cbaff990-6280-11e8-b689-070f60f264ee')/FeatureOfInterest",
    "@iot.id" : "cbaff990-6280-11e8-b689-070f60f264ee",
    "@iot.selfLink" : "http://localhost:8080/FROST-Server/v1.0/Observations('cbaff990-6280-11e8-b689-070f60f264ee')"
  }

Now the question would be: how can i get FoI, if i want to fetch Observations via MQTT?
By querying the navigationLink additionally? Then cache it locally?
Or it there any smarter way?

[hylkevds]

Yes, currently there is no way to automatically get the FoI using MQTT, you'll have to fetch the FoI separately. If the Sensor is not moving, you can cache it and in that case it's not much of an issue. You don't want to repeatedly get the same data any way, so you'd just fetch the FoI when initialising.

If the sensor is moving it's more of an issue, since you'd have to separately query the server for each Observation you receive. I guess that's an oversight when defining the standard. I'll open an issue about it on the SensorThings Github page.