Logging

Data Logging

A mechanism for storing data in SMAP has been provided. The data logger is written as an SMAP driver, but receives information in a zero MQ message sent to a topic prefixed with datalogger/log. The source name is configured at the time the logger starts up and is defined in the driver.ini file. The rest of the topic is extracted and used as the path in SMAP for the data point. If the path already exists, the time series items will be added to the bottom of the list. For example, if our source name is 'test data', and the topic we publish data to is datalogger/log/campus1/building1/testdata, then our data will be posted to time series under campus1/building1/testdata in the source name 'test data'.

The Logger should be added to the sMAP configuration file alongside the Catalyst configuration. The location listed in the configuration file will not be used unless no path is given after datalogger/log.

[/datalogger]
type = volttron.drivers.data_logger.DataLogger
interval = 1

Data Logging Format

Data sent to the data logger should be sent as a JSON object that consists of a dictionary of dictionaries. The keys of the outer dictionary are used as the points in SMAP to store the data items. The inner dictionary consists of 2 required fields and 1 optional. The required fields are "Readings" and "Units". Readings contains the data that will be written to SMAP. It may contain either a single value, or a list of lists which consists of timestamp/value pairs. Units is a string that identifies the meaning of the scale values of the data. The optional entry is data_type, which indicates the type of the data to be stored. This may be either long or double.

{
    "test3": {
        "Readings": [[1377788595, 1.1],[1377788605,2.0]],
        "Units": "KwH",
        "data_type": "double"
    },
    "test4": {
        "Readings": [[1377788595, 1.1],[1377788605,2.0]],
        "Units": "TU",
        "data_type": "double"
    }
}

Data Logger Response

The data logger reports the status of the storage request to topic called datalogger/status. The response will either be success, or error, and the message will contain information on what failed if there was an error. In addition, the data logger looks in the message headers for a field called "from" to determine who should be the recipient of this message. The status message contains a header called "to" that uses the value retrieved from "from" so that agents may filter messages sent to datalogger/status using the match_headers decorator in a base agent derived agent.

Example Code

        headers[headers_mod.FROM] = self._agent_id
        headers[headers_mod.CONTENT_TYPE] = headers_mod.CONTENT_TYPE.JSON
        
        mytime = int(time.time())
        
        content = {
            "listener": {
                "Readings": [[mytime, 1.0]],
                "Units": "TU",
                "data_type": "double"
            },
            "hearbeat": {
                "Readings": [[mytime, 1.0]],
                "Units": "TU",
                "data_type": "double"
            }
        }
        
        
        
        self.publish('datalogger/log/', headers, json.dumps(content))