Skip to content

SolarQuery API global objects

Jump to bottom Edit New page
Matt Magoffin edited this page Jan 31, 2023 · 13 revisions

This page documents structures used throughout the SolarQuery API.

Object Description
Datum stream aggregate A compact aggregate datum used in the SolarQuery Stream API.
Datum stream datum A compact datum used in the SolarQuery Stream API.
Paged query parameters Standard request parameters used for queries that support paginated results.
Paged results object A response structure used for paginated query results.
Sort query parameters Standard request parameters used for queries that support ordering of the results.
Source ID resolving Process by which source ID query parameters are resolved against security token policies.
Virtual ID queries Standard request parameters used for queries that support "virtual" node/source ID results.

Datum stream datum

Datum stream datum are array objects returned from SolarQuery Stream API methods. They are a compact encoding of a datum, without any property names.

Each datum that matches the request criteria is represented as an array with the following items:

  • A reference to the datum stream metadata associated with this datum. The metadata defines the property names and their order within this array, as well as the node/location/source/stream IDs associated with the datum stream.
  • Datum timestamp
  • Instantaneous datum properties, as individual array elements
  • Accumulating datum properties, as individual array elements
  • Status datum properties, as individual array elements
  • Tags, as individual array elements

An example datum stream datum looks like this:

[
  0,
  1650667326308,
  12326,
  null,
  230.19719,
  50.19501,
  6472722,
  "ok",
  "reset"
]

More formally, the datum stream datum array structure looks like this:

Index Type Description
0 integer The 0-based index in the meta array of the datum stream metadata associated with this datum.
1 integer The datum timestamp, in millisecond epoch format.
2..x decimal The instantaneous datum property values, ordered by the meta.i array of the associated datum stream metadata.
(2+iLen)..x decimal The accumulating datum property values, ordered by the meta.a array of the associated datum stream metadata.
(2+iLen+aLen)..x string The status datum property values, ordered by the meta.s array of the associated datum stream metadata.
(2+iLen+aLen+sLen)..x string The tag values, if any tags available.

The iLen, aLen, and sLen variables represent the number of instantaneous, accumulating, and status datum properties (respectively) included in the metadata associated with the datum. null will be inserted for any property defined in the metadata that the datum does not have.

Take the example datum stream datum array shown previously, in the context of a full response with associated metadata:

{
  "success" : true,
  "meta": [
    {
      "streamId": "7714f762-2361-4ec2-98ab-7e96807b32a6",
      "zone": "Pacific/Auckland",
      "kind": "n",
      "objectId": 123,
      "sourceId": "/power/1",
      "i": [
        "watts",
        "current",
        "voltage",
        "frequency"
      ],
      "a": [
        "wattHours"
      ],
      "s": [
      	"state"
      ]
    }
  ],
  "data": [
    [
      0,
      1650667326308,
      12326,
      null,
      230.19719,
      50.19501,
      6472722,
      "ok",
      "reset"
    ]
  ]
}

Given that metadata, the elements of the datum stream datum array represent:

Index Value Description
0 0 the first datum stream metadata returned in the meta array, i.e. stream 7714f762-2361-4ec2-98ab-7e96807b32a6
1 1650667326308 the timestamp in millisecond epoch form, i.e. Saturday, April 23, 2022 10:42:06.308 AM in the Pacific/Auckland time zone given in the stream metadata.
2 12326 the watts instantaneous property
3 null the current instantaneous property (missing in datum)
4 230.19719 the voltage instantaneous property
5 50.19501 the frequency instantaneous property
6 6472722 the wattHours instantaneous property
7 ok the state status property
8 reset a tag

Datum stream aggregate

Datum stream aggregates are array objects returned from SolarQuery Stream API methods. They are a compact encoding of an aggregate datum, without any property names.

Each aggregate datum that matches the request criteria is represented as an array with the following items:

  • A reference to the datum stream metadata associated with this datum. The metadata defines the property names and their order within the datum array, as well as the node/location/source/stream IDs associated with the datum stream
  • A 2-element array with the aggregate start and end timestamps, as millisecond epoch numbers. For queries where the aggregate period is a fixed value, the end timestamp will be set to null as the value is implied by the aggregate period.
  • Instantaneous datum properties, each as 4-element arrays representing the average value, count of values, minimum value, and maximum value.
  • Accumulating datum properties, each as 3-element arrays representing the difference between the ending and starting values, the starting value, and the ending value
  • Status datum properties, as individual array elements. Not all queries return status properties, so these may be omitted.
  • Tags, as individual array elements. Not all queries return tags, so these may be omitted.

An example datum stream datum looks like this:

[
      0,
      [
        1650945600000,
        1651032000000
      ],
      [
        3.6,
        2,
        0,
        7.2
      ],
      [
        19.1,
        2,
        18.1,
        20.1
      ],
      [
        1.422802,
        1138.446687,
        1139.869489
      ],
      "Nominal"
]

More formally, the datum stream aggregate array structure looks like this:

Index Type Description
0 integer The 0-based index in the meta array of the datum stream metadata associated with this datum.
1 array<integer> 2-element array representing the start and end reading timestamps, in millisecond epoch format. ⚠️ For aggregate results from a query with a fixed aggregate period, the end timestamp will be null as the value is implied as the start timestamp plus the requested aggregation period.
2..x array<decimal> The instantaneous datum property values, each as a 4-element array representing the average value, count of values, minimum value, and maximum value, ordered by the meta.i array of the associated datum stream metadata. ⚠️ Not all queries returns instantaneous values; if they are not supported the results will have null instead of a 4-element array.
(2+iLen)..x array<decimal> The accumulating datum property values, each as a 3-element array representing the difference between the ending and starting values, the starting value, and the ending value, ordered by the meta.a array of the associated datum stream metadata.
(2+iLen+aLen)..x string The status datum property values, ordered by the meta.s array of the associated datum stream metadata. ⚠️ These may be omitted if the query does not support status values.
(2+iLen+aLen+sLen)..x string The tag values, if any tags available. ⚠️ These may be omitted if the query does not support tags.

The iLen, aLen, and sLen variables represent the number of instantaneous, accumulating, and status datum properties (respectively) included in the metadata associated with the datum. null will be inserted for any property defined in the metadata that the datum does not have.

Take the example datum stream aggregate array shown previously, in the context of a full response with associated metadata:

{
  "success": true,
  "meta": [
    {
      "streamId": "5514f762-2361-4ec2-98ab-7e96807b3255",
      "zone": "America/New_York",
      "kind": "n",
      "objectId": 123,
      "sourceId": "/pyrometer/1",
      "i": [
        "irradiance",
        "temperature"
      ],
      "a": [
        "irradianceHours"
      ],
      "s": [
        "state"
      ]
    }
  ],
  "data": [
    [
      0,
      [
        1650945600000,
        1651032000000
      ],
      [
        3.6,
        2,
        0,
        7.2
      ],
      [
        19.1,
        2,
        18.1,
        20.1
      ],
      [
        1.422802,
        1138.446687,
        1139.869489
      ],
      "Nominal"
    ]
  ]
}

Given that metadata, the elements of the datum stream aggregate array represent:

Index Value Description
0 0 the first datum stream metadata returned in the meta array, i.e. stream 5514f762-2361-4ec2-98ab-7e96807b3255
1 [1650945600000,1651032000000] the start/end timestamps in millisecond epoch form, i.e. midnight Saturday, Tuesday April 26, 2022 - Wednesday April 27, 2022 in the America/New_York time zone given in the stream metadata.
2 [3.6,2,0,7.2] the irradiance instantaneous property: average of 3.6 from 2 datum that ranged in value from 0 to 7.2
3 [19.1,2,18.1,20.1] the temperature instantaneous property: average of 19.1 from 2 datum that ranged in value from 18.1 to 20.1
4 [1.422802,1138.446687,1139.869489] the irradianceHours accumulating property that changed by 1.422802 from 1138.446687 to 1139.869489
5 Nominal the state status property value

Paged results object

Many API methods can return large sets of data, but SolarQuery will break up the results into smaller paged response objects that contain just a subset of the overall data. The response object contains details on the total number of available results along with the starting offset of the returned results:

{
  "success": true,
  "data": {
    "results": [...],
    "totalResults": 2384790,
    "startingOffset": 0,
    "returnedResultCount": 250
  }
}

In the preceding example, 2384790 results are available, but only the first 250 results have been returned in the data array. To get the next set, or page, of results you'd call the same method again but provide a 0-based starting offset for the desired first result. In this case, an offset of 250 would return the next page, which would result in a response like:

{
  "success": true,
  "data": {
    "results": [...],
    "totalResults": 2384790,
    "startingOffset": 250,
    "returnedResultCount": 250
  }
}

Paged query parameters

For API methods that return paged results objects, the following query parameters are used to specify the page attributes:

max A positive integer for the maximum number of results to return in the response. SolarQuery might limit the number of returned results and ignore this value if it is too large.
offset A non-negative integer for the starting offset of the results to return in the response. When combined with the max parameter you can get individual _pages_ of results from the overall result set.

Example

An example URL query string to return page 2 of a result set using 100 results per page:

?max=100&offset=100

Sort query parameters

For API methods that support sorting the result, an array of sort descriptors can be provided on the sorts query parameter. Array parameters must be specified as sort[x].y where x is a 0-based index that specifies the order of the sort descriptor and y is the sort descriptor property as detailed here:

sortKey The name of the property to sort the results by. Typically this will correspond to the name of an attribute in the returned result objects. For example to sort a location search by name, a query parameter like sorts[0].sortKey=locationName might be appropriate. Consult the documentation for individual API methods for details on the supported sortKey values.
descending A boolean indicating the sort should be in descending (true) or ascending (false) order. By default this value is false so sorts will be applied in ascending order if this is not specified.

Example

An example URL query string to sort by creation in descending order:

?sorts%5B0%5D.sortKey=created&sorts%5B0%5D.descending=true

Source ID resolving

In queries like /datum/list that accept sourceId or sourceIds parameters of a source ID value or pattern (or list thereof), a security policy associated with a security token that has a source ID policy can influence the actual source IDs used in query:

  1. Non-pattern query source IDs that do not match a policy ID (either exactly or from a policy source ID pattern) will be removed from the query.
  2. Query source ID patterns will be removed from the query and then resolved against the policy IDs, with any policy IDs that match added to the query.

Here are some examples to help visualize these rules:

Query Sources Policy Sources Resolved Sources Description
a, b, c b, c, d b, c a is removed because it is not in the policy.
* a, b a, b A pattern query parameter resolves to all matching policy IDs.
/a/** /a/1, /a/2, /b/1 /a/1, /a/2 A pattern query parameter resolves to matching policy IDs.
/a/1, /b/** /a/1, /a/2, /b/** /a/1, /b/** Exact matches (simple and pattern) are preserved.
/a/1, /b/** /a/1, /a/2, /b/1, /b/2 /a/1, /b/1, /b/2 /b/** is resolved to matching policy IDs.

Virtual ID queries

Virtual ID queries are queries where real node or source ID values are combined into results using a virtual ID. Imagine you had several PV inverters on a large solar array, each reporting their energy output on source IDs /pv/1, /pv/2, and /pv/3. You'd like to know the total energy output of all these inverters together. You could query for all 3 source IDs and then add up the results yourself, like this (date parameters omitted):

?nodeId=123&sourceIds=/pv/1,/pv/2,/pv/3&aggregation=Hour

which might return results like this:

[
  {"created":"2018-05-01 12:00:00.000Z", "sourceId":"/pv/1", "wattHours":1000},
  {"created":"2018-05-01 12:00:00.000Z", "sourceId":"/pv/2", "wattHours":1000},
  {"created":"2018-05-01 12:00:00.000Z", "sourceId":"/pv/3", "wattHours":1000},
  {"created":"2018-05-01 13:00:00.000Z", "sourceId":"/pv/1", "wattHours":1000},
  {"created":"2018-05-01 13:00:00.000Z", "sourceId":"/pv/2", "wattHours":1000},
  {"created":"2018-05-01 13:00:00.000Z", "sourceId":"/pv/3", "wattHours":1000}
]

Virtual IDs provide a way to let SolarQuery sum them for you, by mapping the three source IDs into a single virtual ID PV. To do this, you would provide query parameters like this (in addition to those shown before):

?combiningType=Sum&sourceIdMaps=PV:/pv/1,/pv/2,/pv/3&withoutTotalResultsCount=true

This query will return a single result for those three real source IDs, like this:

[
  {"created":"2018-05-01 12:00:00.000Z", "sourceId":"PV", "wattHours":3000},
  {"created":"2018-05-01 13:00:00.000Z", "sourceId":"PV", "wattHours":3000}
]

Node IDs can also be combined into virtual IDs. Imagine a query like this (date parameters omitted):

?nodeIds=123,345&sourceIds=/pv/1,/pv/2,/pv/3&aggregation=Hour

that returned results like this:

[
  {"created":"2018-05-01 12:00:00.000Z", "nodeId":123, "sourceId":"/pv/1", "wattHours":1000},
  {"created":"2018-05-01 12:00:00.000Z", "nodeId":123, "sourceId":"/pv/2", "wattHours":1000},
  {"created":"2018-05-01 12:00:00.000Z", "nodeId":123, "sourceId":"/pv/3", "wattHours":1000},
  {"created":"2018-05-01 12:00:00.000Z", "nodeId":345, "sourceId":"/pv/1", "wattHours":1000},
  {"created":"2018-05-01 12:00:00.000Z", "nodeId":345, "sourceId":"/pv/2", "wattHours":1000},
  {"created":"2018-05-01 12:00:00.000Z", "nodeId":345, "sourceId":"/pv/3", "wattHours":1000}
]

These can be combined into a single -100 virtual ID with query parameters like this (in addition to those shown before):

?combiningType=Sum&nodeIdMaps=-100:123,345&withoutTotalResultsCount=true

that returned results like this:

[
  {"created":"2018-05-01 12:00:00.000Z", "nodeId":-100, "sourceId":"/pv/1", "wattHours":2000},
  {"created":"2018-05-01 12:00:00.000Z", "nodeId":-100, "sourceId":"/pv/2", "wattHours":2000},
  {"created":"2018-05-01 12:00:00.000Z", "nodeId":-100, "sourceId":"/pv/3", "wattHours":2000}
]

Both node IDs and source IDs can be combined at once, so for example you could combine all results into a single virtual source ID PV and node ID -100 with query parameters like:

?combiningType=Sum&nodeIdMaps=-100:123,345&sourceIdMaps=PV:/pv/1,/pv/2,/pv/3&withoutTotalResultsCount=true

which would result in:

[
  {"created":"2018-05-01 12:00:00.000Z", "nodeId":-100, "sourceId":"PV", "wattHours":6000},
]

You can even provide multiple virtual ID mappings, for example to return virtual node IDs -100 and -200 and virtual source IDs PV and LOAD you might have these query parameters:

?combiningType=Sum&nodeIdMaps=-100:123,345&nodeIdMaps=-200:456,678&sourceIdMaps=PV:/pv/1,/pv/2,/pv/3&sourceIdMaps=LOAD:/meter/1,/meter/2&withoutTotalResultsCount=true

Virtual ID combining types

The combiningType query parameter defines the operation performed to combine individual results into combined results. See the supported combining types for reference.

Some combining types might be sensitive to the order the individual results are combined in. For example, the Difference type performs a subtraction operation, so the order is significant. The order is defined by the order of the real IDs defined in each nodeIdMaps and sourceIdMaps parameter value. For example, for these query parameters:

?combiningType=Difference&sourceIdMaps=SelfUse:PvGeneration,GridExport&withoutTotalResultsCount=true

defines a subtraction operation like SelfUse = PvGeneration - GridExport.

Add a custom footer
Clone this wiki locally