[Question] Missing documentation on telemetry API / timeseries data request

[sglux]

Component

• telemetry API / documentation

Description
When using the telemetry API to request historical timeseries data by using
http(s)://host:port/api/plugins/telemetry/{entityType}/{entityId}/values/timeseries?keys=key1,key2,key3 according to this documentation https://thingsboard.io/docs/user-guide/telemetry/#timeseries-data-values-api
it remains unclear how to determine if all requested data has been returned or not.

Let's assume a time range of 24 hours is requested and the limit is set to 1000 but there are in fact more than 1000 values existing in that range.

Now the first request returns exactly 1000 data points, but there is no indication if there are more data in that time range that hove not been returned.

It is also not documented how to request the remaining timeseries data of the given time range, if there are any.

I know from other API requests, that they return a flag telling there are more data and also a counter to be used in subsequent requests to get the rest of the data. Example:
get /api/tenant/devices{?type,textSearch,idOffset,textOffset,limit}
returns helpful information about requests to fire subsequently:

{
  "data": [
    {
      "id": {
        "entityType": "DEVICE",
        "id": "f524fbe0-343a-11e9-a96e-e5f9d91da906"
      },
      "createdTime": 1550576343710,
      "additionalInfo": {
        "description": "Rückstellmuster Colombia UVI,\nCPUID 0xBFEDD923"
      },
      "tenantId": {
        "entityType": "TENANT",
        "id": "d3a77d70-2563-11e9-b9a2-0bf84676a17b"
      },
      "customerId": {
        "entityType": "CUSTOMER",
        "id": "056dd4e0-5605-11e9-92a2-af95e9ea2dfb"
      },
      "name": "Colombia-RSM",
      "type": "sg-iot-gen2",
      "label": null
    }
  ],
    "nextPageLink": {
    "limit": 1,
    "textSearch": null,
    "textSearchBound": null,
    "textOffset": "colombia-rsm",
    "idOffset": "f524fbe0-343a-11e9-a96e-e5f9d91da906"
  },
  "hasNext": true**
}

Questions:

• was this simple forgotten to implement or to document for timeseries data requests?
• is there another solution beside determining from the timestamps of the returned data if there may be more data?
• is an empty response body the only indication that there are no more data in the requested time range? Is it a bullet proof indicator?

Environment

• OS: Ubuntu 20 LTS
• ThingsBoard: 2.5.4 (dockerized, with cassandra database)
• same issue appears via SwaggerUI or Python or whatever way the API is used

[Karabiy]

this API request provides you an option not to set limit. the limit variable is being considered not the required, so that those might not be used.
In case if you want to receive a "batches", you can have the second request invocation to be set for the ts which the last received telemetry record refers for, so that you will need to have one more request to validate whether there is any more data, to be invoked

[sglux]

Well, I have checked with SwaggerUI and also API calls:

• using &limit=10000& gave in this case 5813 points, expected
• using &limit=1000000& gave e.g. 38000 points, expected, still quick
• omitting the &limit=something& part gave me 100 points, which seems to be the default
• using &limit=NONE& or &limit=NONE& gave "Bad Request". This is a little strange, since &agg=None& is accepted as well as &agg=NONE&. So a hint that None is not supported?

Question: how to express "no limit" for this API call?

Looking in TelemetryController.java it seems to me that the default in case the parameter is not given is 100 and the hard limit is what can be encoded in an integer.

[sglux]

Looks like the algorithm to request an unknown number of point in pseudo-code would be:

startTs = sometime
endTs = sometime later
anyLimit = anyLimit
do loop {
     request data from startTs to endTs with anyLimit
     if (number of points returned equals anyLimit){
        set startTs to timestamp of last received data point
     }
     append received points to output (may it be 0...anyLimit number of points)
until (number of received points < anyLimit)

This should work. I still wonder why other requests have the .hasNext feature but not the timeseries request.

Anyway, the limits of the limit parameter could be better documented.

[sglux]

Another question arose: in TelemetryController.java lines 208 and 214 the parameter orderBy is handled which is not exposed in the SwaggerUI.
Why that? Maybe this is worth a separate thread.

[mykhailokornieiev]

As the community has not provided assistance on this matter over an extended period, I am closing this ticket due to inactivity. If it is still relevant, feel free to open a new ticket with a full description of your request.