Incorporating a SensorThingsAPI DataProvider into QGIS core (WIP)

[rduivenvoorde]

QGIS Enhancement: Incorporating a SensorThingsAPI DataProvider into QGIS core (WIP)

Date 2022/10/30

Author Richard Duivenvoorde (@rduivenvoorde) and others

Contact richard@duif.net

maintainer @rduivenvoorde and others

Version QGIS master

Summary

Sensor Things API (STA in short), is an OGC specification for storing and retrieving (timeseries) of Sensor Observations.

The Frost-Server is one of current default implementations: https://github.com/FraunhoferIOSB/FROST-Server with a very good documentation site with a lot of background information: https://fraunhoferiosb.github.io/FROST-Server/sensorthingsapi/1_Home.html
There is also: https://developers.sensorup.com/docs/#sensorthingsAPISensing

A STA server can be requested for 'STA-objects' or FeaturesOfInterest via REST http requests. STA has a query language based on OASIS Odata URL patterns and query options.

There is a QGIS Client plugin: https://plugins.qgis.org/plugins/SensorThingsAPI/ which you can use to to talk to a STA server ( eg https://airquality-frost.k8s.ilt-dmz.iosb.fraunhofer.de/v1.1/Locations with European Air Quality data.)

The plugin has an onboard python(!) QGIS DataProvider: https://github.com/AirBreak-UIA/SensorThingsAPI_QGIS-plugin/blob/main/providers/provider_frost.py which is currently only able to request Locations as Geometrical objects/features, and after that will request timeseries observation data from one Location

Current QEP is about extending this DataProvider to request other STA Objects or FeaturesOfInterest (FOI's), and either pull the Python provider into core or port it to C++ and pull it into core.

My Section

STA has several objects. The datamodel is fixed to the following:

As said earlier: current plugin only requests the Locations, and then further queries the properties of ONE location. This usecase fits best to fixed observation locations, of which one is interested in the data of one location.

Another usecase more fitting into the QGIS feature model, is that one requests the Observations of 1 Datastream, including their Locations. In that way we could show the Locations with Values of given TimeRange, and create more 'GIS'-like data: a Feature with a Geometry and Attributes.
In that way we could also couple the provider to the Temporal Controller and request TimeFrames of data.
This usecase is better suited for data in which you want to have an spatial(!) overview of your data (in time) (for example the dispersion of some chemical over an area)

Last usecase is where One Thing (one vehicle with several Sensors) is driving and one wants to view all the data of the vehicle in time during the ride (in STA terms: you would retrieve the Features Of Interes (with Historical Locations) of one Thing, with the observation(s).

Proposed Solution

Question here is what is the best way forward: further implement the provider (and a simple starting GUI) in Python.
Or is it better to implement it in C++

Example(s)

Current Provider in Python: https://github.com/AirBreak-UIA/SensorThingsAPI_QGIS-plugin/blob/main/providers/provider_frost.py

Affected Files

DataProviders and simple GUI files. The idea is that users will create/use other Python plugins/GUI's to further make their usecases work

Performance Implications

As other services, requesting features from servers, there is the issue that a user can request too many features at once.

Further Considerations/Improvements

?

Backwards Compatibility

None

Issue Tracking ID(s)

None

Votes

[troopa81]

Question here is what is the best way forward: further implement the provider (and a simple starting GUI) in Python.
Or is it better to implement it in C++

It's an OGC standard so I think it makes sense to implement it in C++ in core

[KathiSchleidt]

It would be great if such functionality would also support extensions to the STA data model, e.g., the polymorphism we applied to STA Thing to support the various spatial featureTypes under INSPIRE Soil.

• Details at: https://github.com/hylkevds/FROST-Server.SoilThings
• Demo-Endpoint realizing this extended model at: https://ogc-demo.k8s.ilt-dmz.iosb.fraunhofer.de/FROST-SoilThings/v1.1/

In addition, would be nice if this can cope with the STA-Plus model, to my understanding slated as an OGC community extension.

[sgrellet]

That would be indeed a great addition to QGIS
We are currently running a Water Quality OGC Interoperability Experiment and expect several data flows to be exposed using ST API (see https://github.com/opengeospatial/WaterQualityIE )

We will be able to provide support to this exercise (funding, endpoints, ...)

[AirBreak-UIA]

Possible ideas for refactoring and improving the existing experimental plugin approach.
Focus on extending the access to STA endpoints considering not only LOCATIONS but also other elements of STA:

1. Query DISTINCT values of the following STA elements:
THINGS

• SELECT attributes to show
• FILTER attributes’ values

LOCATIONS

• [select, filter…]
• SPATIAL FILTER (bbox, other layer’s extent, etc)

HISTORICAL LOCATIONS

• [select, filter…]
• TIME FILTER (interval, etc)

SENSORS

• [select, filter…]

OBS PROPERTIES

• [select, filter…]

MULTI / DATASTREAMS

• [select, filter…]

FEATURES OF INTEREST

• [select, filter…]
• SPATIAL FILTER (bbox, other layer’s extent, etc)

OBSERVATIONS

• [select, filter…]
• TIME FILTER (interval, etc)

2. Chose element to use as geometry on map (either Locations or FoIs).
The other is requested will be provided as a geojson string.

3. Choose data structure for output: tabular or hierarchical

[AirBreak-UIA]

Simplified sub-set of the previous controls, for specific query use cases:

• Moving sensors: querying just Locations (*map), Things and Location timestamps
• Things map: querying just Locations (*map), Things, Multi/DS and Obs Properties
• Areas of Interest map: querying just FoIs (*maps) and related Things, Multi/DS and Obs Properties
• Multi/DS time series
• Etc… maybe configurable