SolarNode Datum Throttling

Managing SolarNode data costs

This guide describes how you can reduce the amount of bandwidth SolarNode requires and how much data SolarNode uploads to SolarNetwork. This could be important to you because of a variety of reasons:

• The node is on a metered internet connection, like a mobile connection, and the data cost is expensive.
• Various node data sources produce more data properties than you need, and you'd like to reduce the associated costs incurred by your SolarNetwork subscription.
• You would like to collect high-frequency data on the node for use on the node itself, but only want to upload a lower-frequency set of data to SolarNetwork.

Use SolarIn/MQTT

There are two main ways SolarNode can post data to SolarNetwork:

1. The traditional periodic bulk upload via the SolarIn/HTTP API.
2. The real-time SolarIn/MQTT API.

The SolarIn/MQTT API typically requires less bandwidth overall than the SolarIn/HTTP API, because:

1. The HTTP API requires TLS negotiation on every short-lived upload transaction, while the MQTT API uses long-lived connections that require negotiation only at the start.
2. The HTTP API encodes the data using JSON, while the MQTT API uses the more space-efficient CBOR encoding.

The current SolarNodeOS images and SolarNode core application packages published by the SolarNetwork Foundation have SolarIn/MQTT included and configured by default. If you have an older SolarNode deployment, or are rolling your own SolarNode packages, then you might not be including SolarIn/MQTT support.

Use Datum Property/Throttle filters

The Datum Sample Property Filter SolarNode plugin provides two services that allow you to control exactly which datum properties are uploaded to SolarNetwork and to throttle the frequency of properties or entire sources.

Reduce the number of datum properties uploaded

The Datum Property Filter allows you to configure include/exclude rules on which datum properties are uploaded to SolarNetwork. For example, imagine a 3-phase power meter data source plugin that produces datum with lots of interesting properties like:

• apparentPower
• current
• frequency
• reactivePower
• powerFactor
• voltage
• wattHours
• watts

If you only have a need for the watts and wattHours properties, you could configure a Datum Property Filter that specifies only those two properties should be posted to SolarNetwork. That would reduce the associated per-datum property cost by 75%.

This filter also lets you throttle the frequency of individual properties, in case you want to collect two properties from the same source at different rates. For example, you could configure the data source to collect datum once/minute, but then throttle the watts property to once/5-minutes.

Throttle datum frequency

The Datum Throttle Filter allows you to reduce the frequency at which a data source posts datum to SolarNetwork. This effectively slows down the rate at which a source uploads data. SolarNode will still collect data from the filtered sources at whatever rate is defined on the source, but it will limit the upload of datum to at most the rate configured on the filter. This can be useful if other plugins on the node_ still require access to the higher-frequency data but you only require a lower-frequency rate saved to SolarNetwork.

Configure an expiration policy

All datum posted to SolarNetwork stay and accumulate in SolarNetwork, which incurs a storage cost. On SolarNetwork you can configure expiration policies to delete old datum. This will keep a limit on the overall storage costs.