Timeseries Data
The Omnia Timeseries API can be used to retrieve raw timeseries data points (raw = as they are processed/persisted in the source/publisher, i.e. the entity that streams and/or writes timeseries data to the Omnia Plant platform).
Data point can be returned either in JSON or Protobuf. For protobuf see https://github.com/equinor/OmniaPlant/wiki/ProtoBuf.
To successfully retrieve timeseries data, the client must as a minimum specify the unique id of the timeseries (automatically assigned when creating metadata object). You should also specify a given time window (start and end time) in the API request. If you don't include time parameters, the time window will be the full history available.
The API currently supports the following methods:
- Get raw and aggregated data points (supports paging).
- Get raw and aggregated data points from multiple timeseries in one request (supports paging) with independent time window and filter per timeseries.
- Get the first data point of either one timeseries or multiple.
- Get the latest data point of either one timeseries or multiple.
- Write new data points.
- Delete data points (hard delete, no soft-delete on datapoints).
Get first and last data point of a timeseries gives the client information about the amount of historical data on a timeseries.
A single data point returned from the API will consist of a timestamp, value and status. The status flag represents the quality state for a data point. The reliability of this status flag depends on the reliability and how this value is handled in the sources publishing timeseries data to Omnia Plant Data Platform. See status values and mapping from main source values below.
| Status | API Value | mapping from: | PI | IP21 |
|---|---|---|---|---|
| Good | 192 | 192 | 0 | |
| Questionable | 64 | 64 | 1 | |
| Bad | 0 | 0 | 2 | |
| Substituted | 24 | 24 | 4 | |
| UncertainSubstituteValue | 8 | 8 | 5 | |
| BadSubstituteValue | 60 | 60 | 6 | |
| Good but Null | 256 | n/a | 0 with null value |
When requesting timeseries data (both raw and aggregated) you filter on the status. If you e.g. filter on status = 192 only datapoints with status = 192 will be returned. Use filter on status with care, as it reduces the request performance.
The Omnia Timeseries API currently only supports values of type Double.
The sample data points in the below table are used to describe and exemplify the rest of the API features.
| Time | Value |
|---|---|
| 2019-02-05T15:00:00.000Z | 1 |
| 2019-02-05T15:00:05.000Z | 2 |
| 2019-02-05T15:00:10.000Z | 3 |
| 2019-02-05T15:00:15.000Z | 4 |
| 2019-02-05T15:00:20.000Z | 5 |
| 2019-02-05T15:00:25.000Z | 6 |
When requesting raw data points within a given time window, the client can optionally specify a includeOutsidePoints boolean (true or false) that determines whether or not the data points immediately before (left outside) and after (right outside) the time window shall be included in the result. When querying with outside points. We limit how far outside we search equal to the size of the time window used in query, ie when searching for 1 hr of data, we look 1 hr before for first left outside point, and 1 hr after for first right outside point. This can be useful for interpolating data. Use includeOutsidePoints with care, as it reduces the performance of the request.
The following request will return the data points with value = 2, 3 and 4 which are within the time window. Since includeOutsidePoints is set to true, the API will also return the data point with value = 1 since it is the first data point before the given time window, and it will return the data point with value = 5 since it is the first data point after the given time window.
GET raw data points between 2019-02-05T15:00:05Z and 2019-02-05T15:00:15Z with includeOutsidePoints = true
When requesting raw data points within a given time window, the default limit in the response is 1000. If a timeseries has 5000 data points in the database from time A to time B, the corresponding API request will return 1000 data points if not any other limit is specified. The response will also include a continuation token that indicates there are more data points on the given timeseries within the requested time window.
The client can optionally specify a limit in the API request that is the limit of data points to retrieve within the given time window. For timeseries data, this specified limit must be between 1 and 100 000.
The following query will return the first two data points with value = 1 and 2. The response will also include a continuation token that indicates more data on the timeseries within the given time window. If the client adds the continuation token as a query parameter in the next API request, the API will return the next two data points with value 3 and 4.
GET raw data points between 2019-02-05T14:00:00Z and 2019-02-05T16:00:00Z with limit = 2
Paging and inclusion of outside data points can be combined in a single request.
The following request will return the last data point before the specified start time (value = 1), and the data points with value = 2 (timestamp: 2019-02-05T15:00:05.000Z) and value = 3 (timestamp: 2019-02-05T15:00:10.000Z). This response will also include a continuation token since there are more data points on this timeseries within the requested time window. The right outside point (which in this example would have been the data point with value = 4) is therefore not included in this response. Use the token instead to get the next result set.
GET raw data points between 2019-02-05T15:00:05Z and 2019-02-05T15:00:15Z with limit = 2 and includeOutsidePoints = true
When the client adds the continuation token as a parameter in the next request, the API will return the data point with value = 4 (timestamp: 2019-02-05T15:00:15.000Z) which is the last data point within the requested time window. Since the request asks for outside data points as well, this second response will also include the right outside data point which is the one with value = 5 (timestamp: 2019-02-05T15:00:20.000Z)
The federation concept allows the consumer to read data that is not replicated into the Omnia Timeseries container. Different sources may be provided (IMS, DataLake).
Federation shall be used with care, typical use cases will be:
- Read data older than those provided in Omnia Timeseries container (typically older that 2 years).
- Read data from timeseries that is currently not replicated to Omnia Timeseries container.
Data from "on premise" historians.
- federation is not implemented on plants using Bazefield as IMS/historian
- limit on data/aggregates not supported
- fill on data/aggregates not supported, will act as fill none
- data/aggregates, only one aggregate per request supported
- filter on status when federated source is PI historian
- time span limitations, 7 days for raw, 30 days for aggregated
- compact (protobuf) not supported
- first and last when federated source is IP21 historian, time range must be given
- start and end time should be aligned to aggregate interval
- misaligned aggregate window can produce unexpected results
Data from DataLake.
- Aggregates
- limit on aggregates not supported (everything will be returned on first request)
- compact (protobuf) not supported
With the Omnia Timeseries API you can use HTTP POST to add new or update existing timeseries data, if you have been granted access to do so. The API supports writing data to a single or multiple timeseries in one API request. If you write data on exact timestamps that already exist for that timeseries in the Omnia Timeseries DB, the data will be overwritten.
You have the option of choosing to add or update timeseries data asynchronously or not (async=true/false).
Request with async=false means that the data points in the body will be added or updated directly to Omnia Timeseries DB, and you will get an API response when the data is committed in the Omnia Timeseries DB. If the request fails, it is your responsibility to retry and implement logic to make sure data is not lost.
Request with async=true means that you will only get a 202 Accepted response after a permission check is executed. This means that the Omnia Plant Data Platform has accepted the data, and will take the responsibility to process it and add or update the Omnia Timeseries DB. The Omnia Plant Data Platform will ensure data is not lost.
If you are using this API to add/update data continuously or add/update large amounts of data we recommend using async=true to avoid waiting on the commit confirmation. If it is important for you to wait on the response and see if the write was success or not, we recommend using async=false.
Status codes, see list of valid codes (Good = 192).
The API currently only supports writing values of type Double.