-
Notifications
You must be signed in to change notification settings - Fork 5
FAQ
Q: I have a project where we are going to share timeseries ("sensor data") data externally, e.g. with partners and/or suppliers. Which solution do you recommend?
A: We need to understand the use case and needs, but in general we recommend the Omnia Timeseries API as a base case because:
- The API is secured by design and implemented according to our information security requirements.
- The API is internet-facing and made for public use, it works equally for external clients as it does for internal clients. The API can be accessed from any platform and software environment as long as you have access to Internet. There is no need for proprietary point-to-point integrations over VPN etc. forcing both parties to implement licensed and vendor-specific software.
- The API itself is managed as a key product. It means we put high focus on the customers of the API (e.g. developers) trying to make their user experience as best as possible. The API is versioned and is continuously improved based on user feedback and needs.
- The API don't expose any inner workings, implementation details or internal vendor-specific naming schemes from the underlying technology. We can then ensure a continuous optimization of the backend without breaking all clients using the API (agility). We can also start focusing on industry collaboration and interoperability, many companies have different software solutions, and you can't collaborate on common information models with the mindset "we use vendor software A, so it has to be like that", and "we use vendor software B, so it has to be like that".
- To use the API, you don't need any detailed knowledge about the underlying software system. Different plants use different software systems and the Omnia Timeseries API will give you access to timeseries data in one interface independent of plant and software system.
- The API is a full CRUD API, you can create, read, update and delete. The API supports operations like filtering, wildcard search, data aggregations out of the box.
- APIs are a core element of any digital business platform. It is the interface between applications, data and services. Equinor is heading in a "API first" direction - we require software components to provide APIs to communicate with other components, share data and functionality - this is anchored in governing requirements and our API strategy available at https://github.com/equinor/api-strategy
- APIs can be re-used across the organization, internally and externally, meaning that each development team don't have to build processing capabilities from scratch. APIs are increasing the efficiency in software development.
- The API is discoverable in the Equinor API portal (api.equinor.com)
- The API is specified/documented using OpenAPI.
Q: How do I get access to the Omnia Timeseries API?
A: Take a look at https://github.com/equinor/OmniaPlant/wiki/Authentication-&-Authorization
Q: Do I need an Equinor account to use the API?
A: To authenticate towards the API with user impersonation, i.e. a software authenticates on behalf you as a person, you need an Equinor account. For service-to-service (m2m) you don't need any Equinor account. You only need access to Internet and include the client credentials (client id and client secret) that we provide you.
Q: Can I use my own app registration to access the API?
A: You can request user_impersonation for your app registration to access the Timeseries API with your own user permissions. For service-to-service access, you can only use app registrations provided by the Omnia IIoT Team.
Q: I'm having trouble authenticating with the API. Where can I find help?
A: The Omnia Timeseries API uses OAuth 2 for authentication and authorization. The easiest way to use it is using a library such as MSAL which has SDKs for most programming languages. A few examples are also given in Authentication & Authorization. Also, make sure that you are using one of the supported flows, and using the correct app registration information.
Q: To get data, I need an id for a timeseries, but where can I find it?
A: To find timeseries you have access to, use the list timeseries or search timeseries endpoints, see API examples.
Q: What is the publishing rate on the timeseries data in the Omnia Timeseries API, i.e. how often is the data refreshed?
A: For timeseries data originating from the on-premise Historians, the publishing rate is less than 2 minutes, i.e. data is refreshed every 2 minutes.
Q: In the Omnia Timeseries API, will I find all timeseries ("tags") that exist in the on-premise Historians?
A: No, timeseries ("tags") are made available in the Omnia Timeseries DB and Omnia Timeseries API case by case, so it is use-case driven. That said, in the Omnia Timeseries API you will also find data that don't exist in the on-premise Historians, such as new derived and calculated data from projects.
Q: How much history do you make available in the Omnia Timeseries API?
A: We currently don't have any retention policies in the Omnia Timeseries DB. For timeseries data coming from the on-premise Historians we backfill 730 days / 2 years of history when we start streaming the micro batches to Omnia. The backfill is running continuously.
Q: Do the timeseries data need to be in Omnia and the Omnia Timeseries DB to get it from the Omnia Timeseries API?
A: Yes, the data must be in the Omnia Timeseries DB to get it from Omnia Timeseries API. In version 1.7 the federation concept was introduced. This allow access to timeseries data not stored in Omnia, reading directly from on-premise historians.
Q: What is timeseries federation?
A: The federation concept allows the consumer to read data that is not replicated into the Omnia Timeseries container/database. It was introduced in version 1.7. Federation is supported for plants that use PI or IP21 as process historian.
Q: What happens in Omnia Timeseries API if timeseries metadata (name, description, unit) changes in the source system?
A: If a timeseries name ("tag name") is changed (renamed) in the on-premise Historians (OSISoft PI and AspenTech IP.21), the corresponding name will be updated in Omnia and be reflected in the Omnia Timeseries API. The same goes for description and unit. We are not able to automatically detect if someones decides to do the renaming by creating a new timeseries ("tag") in the on-premises Historian. This new tag must be treated as a new timeseries and work must be done to make it and its data available in Omnia. When customers write new timeseries data to Omnia Timeseries API, it is their responsibility to keep metadata updated.
Q: Will a timeseries have the same unique id in both Beta/Test and GA/Prod?
A: No, they will have separate id's.
Q: What does the attribute "terminal" mean?
A: On some facilities using IP21 as process historian, tags may contain multiple datasets, where each dataset must be identified with the attribute "terminal". Inside IP21 this attribute is similar to "map". This extra attribute is needed to identify datasets on tags from Snorre A, Snorre B, and electro-system at Kårstø. If no terminal is indicated, the "default" terminal, if set on source (IP21), will be used.
Q: Do I get the exact same timeseries with data points in both the Omnia Timeseries API and the Data Lake?
A: No, not necessarily. When the timeseries data is coming from the on-premise Historians, i.e. the source is the same, making data available in Data Lake and Omnia Timeseries DB w/ Omnia Timeseries API is two independent services. You may have timeseries in Data Lake which is not available in Omnia Timeseries API and vice versa.
Q: When shall I use the Data Lake and when shall I use the Omnia Timeseries API?
A: The Omnia Timeseries API ("online") is an interface to access the Omnia Timeseries DB. The Omnia Timeseries DB is optimized for large volumes of timeseries data with the ability to perform near real-time queries and analysis on those data. The Omnia Timeseries API therefore supports fast random ad-hoc queries, e.g. among many distinct timeseries I want to see data on Timeseries A, B and C from 12:00 to 12:15. The Omnia Timeseries API also gives you the ability to get aggregated data (max, min, avg, stddev and count) out of the box. A typical primary measure for the Omnia Timeseries API is response time, i.e. for how long do you need to wait for a response on your API request.
The Data Lake on the other hand is for batch processing ("offline") where you take a large amount of input data (e.g. 10 years of timeseries data from a huge number of distinct timeseries), runs a job to process it and produce some output data. These jobs take a while, from hours to days. A typical primary measure for the Data Lake is not response time, but throughput (the number of timeseries and volume of timeseries data we can process per time unit, or the total time it takes to run a job on a dataset of a certain size). When reading files in a distributed file system you scan the entire file which is inefficient if you only want to access a short timespan on a specific timeseries (this is what the Omnia Timeseries API is designed for). Accessing timeseries data in the Data Lake is typically used for training of machine learning models, i.e. batch processing.
Q: Is there an asset model / equipment hierarchy API that can be used to easily find and understand the timeseries data?
A: No, the Omnia Timeseries API itself only provides "flat" timeseries objects without any other context than the name, description, engineering unit and the plant the timeseries is linked to.
Q: I have received client id, client secret, resource id and tenant id, but I get a 401 - "Unauthorized" message when trying a GET request from Postman, what is wrong?
A: Please follow our Postman guide: https://github.com/equinor/OmniaPlant/tree/master/Postman - make sure your credentials are correctly inserted.