SolarNode Local Storage

SolarNode local data storage

The SolarNode runtime provides an embedded data store that is used to hold application settings, data sampled from devices, or anything really. Some data is designed to live only in this local store (such as settings) while other data eventually gets pushed up into the SolarNet cloud. This document describes the most common aspects of the local data store.

Data store implementation

The data store is implemented using the Apache Derby embedded SQL database. Typically the database is often configured to run entirely within RAM on devices that support it, and the RAM copy is periodically synced to non-volatile media so if the device restarts the persisted copy of the database can be loaded back into RAM. This pattern works well because

1. Non-volatile media access can be slow (e.g. flash memory)
2. Non-volatile media can wear out over time from many writes (e.g. flash memory)
3. Aside from settings, which change infrequently, data stays locally only a short time before getting pushed into the SolarNet cloud.

Low level access: JDBC

Since the database is implemented by Derby, a full JDBC stack is thus available and normal SQL queries are used to access the database. The net.solarnetwork.node.dao.jdbc bundle publishes some core JDBC services for other bundles to use, such as:

• A connection-pooled javax.sql.DataSource for direct JDBC access
• A Spring org.springframework.jdbc.core.JdbcOperations for slightly higher-level JDBC access
• A Spring org.springframework.transaction.PlatformTransactionManager for JDBC transaction support

High level access: Data Access Object (DAO)

The SolarNode runtime also provides some Data Access Object (DAO) services that make storing some typical data easier:

• A net.solarnetwork.node.dao.SettingDao for access application settings/configuration/preferences stored as Setting objects

• A DatumDao suitable for storing general node Datum objects

• A DatumDao suitable for storing general location Datum objects

For example, to use the GeneralNodeDatum DAO from a plug-in, using OSGi Blueprint you'd define a reference in your Blueprint XML like this:

<reference id="generalNodeDatumDao"
		interface="net.solarnetwork.node.dao.DatumDao"
        filter="(datumClassName=net.solarnetwork.node.domain.GeneralNodeDatum)"/>

Or to access the Setting DAO, define a Blueprint XML reference like this:

<reference id="settingDao" interface="net.solarnetwork.node.dao.SettingDao"/>

General datum

When collecting data from devices, the most common scenario is to turn that data into instances of the GeneralNodeDatum class. That class provides a [GeneralDatumSamples][GeneralDatumSamples] property that allows arbitrary data values to be persisted, categorized into three groups:

1. instantaneous data, representing "snapshot" numeric measurements such as temperature, electric current, etc.
2. accumulating data, representing counter-like numeric measurements such as a watt-hour meter or odometer
3. status data, representing string messages such as error codes

SolarNetwork treats these categories of data differently, for example when aggregating datum over time the instantaneous values can be be averaged while accumulating values are differenced.

When writing a SolarNode plug-in that samples data from a device, you decide which category to put each sampled value by calling one of these methods:

// pretend we just collected some data from a power meter
String sourceId = "Meter1";
Double wattReading = 2250.0;
Long wattHourMeterReading = 372909820990L;
String operatingMode = "Normal";

// now translate those values into GeneralNodeDatum
GeneralNodeDatum datum = new GeneralNodeDatum();
datum.setCreated(new Date());
datum.setSourceId(sourceId);
datum.putInstantaneousSampleValue("watts", wattReading);
datum.putAccumulatingSampleValue("wattHourReading", wattHourMeterReading);
datum.putStatusSampleValue("opMode", operatingMode);

If you wanted to express that datum object as JSON, you might get:

{
	"created" : "2016-08-17T01:00:00.000Z",
	"sourceId" : "Meter1",
	"instantaneous" : {
		"watts" : 2250.0
	},
	"accumulating" : {
		"wattHourReading" : 372909820990
	},
	"status" : {
		"opMode" : "Normal"
	}
}

That's pretty much the way SolarNetwork sees your datum as well, except that the categories are kind of conceptual, and a better way of thinking of this in JSON would be:

{
    "created"         : "2016-08-17T01:00:00.000Z",
    "sourceId"        : "Meter1",
    "watts"           : 2250.0,
    "wattHourReading" : 372909820990,
    "opMode"          : "Normal"
}

Identity data

There are two other important pieces of data SolarNode stores:

1. certificate key store
2. identity metadata

The certificate key store is a standard Java key store, by default saved to conf/tls/node.jks and this holds the node's private key and signed X.509 certificate.

The identity metadata is a JSON file by default saved to conf/identity.json and this holds metadata like the unique node ID and the password for the key store. An example looks like:

{
  "nodeId" : 163,
  "confirmationCode" : "23989895965aec8c2d8",
  "solarNetHostName" : "data.solarnetwork.net",
  "solarNetHostPort" : 443,
  "solarNetForceTls" : true,
  "keyStorePassword" : "secret"
}