Technical infrastructures
We have deployed three different types of monitoring equipment in homes. In each case, the equipment consists of several electricity sensors connected to a hub (usually a computer or embedded device), where the hub is connected to the home LAN. The hub collects data from the sensors and passes the data to a central MySQL database (via a web-service) on a virtual server (an Amazon AMI) hosted by the Amazon EC2 service. The virtual server also includes an Apache web server, allowing users to view the central database in various representations through a web-browser (either desktop or mobile) and mobile client.
This is equipment from Green Energy Options (http://www.greenenergyoptions.co.uk). The equipment allows both electricity circuits (a group of sockets, or major appliances) and individual sockets to be monitored. Typically, the Green Energy Options hub temporarily stores energy data for "real-time" visualisations on the hub, then sends the data to a central database hosted by Green Energy Options for historical visualisations. We have altered the hub so that the data is sent to our central database. We currently have this type of kit in 4 houses. Data from these kits allow us to visualise multiple streams of data from different electrical appliances.
- The equipment provides three resolutions of data: individual appliance, circuits and total load (the total electricity use can easily be calculated by summing the value of all the circuits). This is not common in commercial home energy monitoring systems.
- The smart plugs used to monitor individual sockets can also control sockets (switch on or off) remotely.
- The equipment is expensive compared to other domestic electricity monitoring solutions: to monitor all circuits and 10 individual appliances in one house costs approximately £1500. This cost is so high because the hub is expensive (the hub is actually a laptop) and an electrician is legally required to install the circuit sensors (labour costs).
- The hub is a laptop which must be turned on constantly; 20W constant use is equivalent to leaving a low-energy lightbulb switched on constantly.
- The software installed on the hub is over-complicated and Green Energy Options are keen to be involved in any changes we make, therefore it takes a long time to modify the hub.
This equipment is from OWL (http://www.theowl.com). It allows the total electricity use of a house to be monitored. It consists of a CT clamp (which is attached to the mains electricity cable where it enters the house) with a wireless transmitter and a wireless receiver with USB adapter. The CT clamp sends electricity use data to the wireless receiver which is connected to a computer in the home. We have written an application to send the data to our central database. We currently have 2 of these kits, but only 1 is installed in a house. Due to the limitations with drivers, we plan to replace this with CurrentCost equipment.
- A single kit is cheap (approx. £80).
- We can only find Windows drivers for the wireless receiver: this means that the home must have a Windows PC in order to send any data to the database.
- The data is sent to the database from a computer, therefore the home must keep a computer turned on constantly in order to provide real-time data. This is expensive.
This equipment is from CurrentCost (http://www.currentcost.co.uk). Similar to the OWL, the equipment uses a CT clamp to monitor the total electricity use of a house. It also has a wireless receiver with USB adapter that can be connected to a computer to allow electricity data to be sent to the web. We have written an application to push the data to our central database. Unlike the OWL, additional sensors can be purchased for the CurrentCost kit to allow individual appliances to be monitored as well as the house total. Also unlike the OWL, there are Linux drivers for the CurrentCost USB adapter: this means that we can use low-power Linux computers (such as the Sheeva Plug (http://www.newit.co.uk/)) to push the data to our central database rather than using a larger home computer. We currently have 36 of these kits. Only 4 of these are currently deployed in homes. 1 of the kits has 9 additional individual appliance monitors, while the remaining 12 only have the total electricity consumption monitors.
- A single kit is cheap (approx. £150 including a Sheeva Plug).
- A Sheeva Plug must be switched on constantly, but this is very low power (approx. 5W maximum) in comparison to the Green Energy Options or OWL hub computers.
- The basic kit may be extended to include individual appliance monitors (approx. £80 each).
We have built a virtual server image (an "Amazon AMI") based on Fedora Linux that includes a MySQL server, Apache web server, Apache Tomcat server and PHP. Amazon EC2 currently hosts 1 instance of this virtual server with a static IP address (79.125.20.47). We leave this server running constantly; it is based on Amazon's free server, so it has fairly limited capabilities, but it currently appears to be sufficient.
The database consists of several tables. The most important are described below.
Every hub that pushes data to the database requires a row in the tblHubDir table. The example below shows a hub called "Leygue Geo Hub" which was deployed at 9AM on 04/01/2011 and is still active (and so has not been decommissioned).
mysql> select * from tblHubDir where hubId=12;
| hubId | description | commissioned | decommissioned |
| 12 | Leygue Geo Hub | 2011-01-04 09:00:00 | NULL |
Every sensor that pushes data to a hub requires a row in the tblSensorDir table. The example below shows all sensors connected to "Leygue Geo Hub". E.g. the first sensor is monitoring "Lights", it was deployed at 8AM on 04/01/2011, it is still active, and it is a category 2 sensor (category 1 means a sensor monitoring all mains electricity used by a home, category 2 means a sensor monitoring a specific electrical circuit within a home, category 3 means a sensor monitoring a specific appliance within the home).
mysql> select * from tblSensorDir where hubId=12;
| hubId | sensorId | description | commissioned | decommissioned | category | parentId |
| 12 | 1 | Lights | 2011-01-04 08:00:00 | NULL | 2 | NULL |
| 12 | 2 | Lights / Smoke Alarm | 2011-01-04 08:00:00 | NULL | 2 | NULL |
| 12 | 3 | Boiler | 2011-01-04 08:00:00 | NULL | 2 | NULL |
| 12 | 5 | Sockets | 2011-01-04 08:00:00 | NULL | 2 | NULL |
| 12 | 6 | Shower | 2011-01-04 08:00:00 | NULL | 2 | NULL |
| 12 | 7 | Kitchen Sockets | 2011-01-04 08:00:00 | NULL | 2 | NULL |
| 12 | 13 | Toaster | 2011-01-04 08:00:00 | NULL | 3 | NULL |
| 12 | 14 | Kettle | 2011-01-04 08:00:00 | NULL | 3 | NULL |
| 12 | 15 | Microwave | 2011-01-04 08:00:00 | NULL | 3 | NULL |
| 12 | 16 | Iron | 2011-01-04 08:00:00 | NULL | 3 | NULL |
| 12 | 17 | Laptop | 2011-01-04 08:00:00 | NULL | 3 | NULL |
A hub pushes electricity usage data into the tblLoad table. The example below shows all electricity data pushed from the "Sockets" sensor to "Leygue Geo Hub" hub in the last 15 seconds. At the four times (6:39:43PM, 6:39:47PM, 6:39:51PM and 6:39:55PM) when the sensor reported data, 27.00 Watts of electricity were being used (e.g. similar to a laptop being charged).
mysql> select * from tblLoad where hubId=12 and sensorId=5 and timeStamp>=(now() - interval 15 second);
| hubId | sensorId | timeStamp | load |
| 12 | 5 | 2011-07-01 18:39:43 | 27.00 |
| 12 | 5 | 2011-07-01 18:39:47 | 27.00 |
| 12 | 5 | 2011-07-01 18:39:51 | 27.00 |
| 12 | 5 | 2011-07-01 18:39:55 | 27.00 |
Once every hour the server is scheduled to compress and move all data from the tblLoad table into the tblLoadAgg table. This means that tblLoad table holds high resolution data for the last hour, while the tblLoadAgg table holds all older data (data older than an hour) at a lower resolution. The example below shows how data from the tblLoad table is compressed into much longer time intervals: in this case there are three records covering a 15 minute period, where the "Sockets" sensor observed an average of 26W being used between 6:25PM and 6:30PM, 26.23W being used between 6:30PM and 6:35PM and 27W being used between 6:35PM and 6:40PM.
mysql> select * from tblLoadAgg where hubId=12 and sensorId=5 and timeStamp >= "2011-07-01 18:30:00" and timeStamp <= "2011-07-01 18:40:00";
| hubId | sensorId | timeStamp | loadAvg |
| 12 | 5 | 2011-07-01 18:30:00 | 26.00 |
| 12 | 5 | 2011-07-01 18:35:00 | 26.23 |
| 12 | 5 | 2011-07-01 18:40:00 | 27.00 |
Every user who wishes to access de-anonymised data from the database or push data into the database requires a row in the tblUser table. In the example below a user with the email address "test@caware.com" has hashed api key and password stored in the database.
mysql> select * from tblUser where email="test@caware.com";
| userId | apiKey | email | password | lastActivity | jitterCount |
| 17 | 6e63a4c76e7feg456461e9b2519703933 | test@caware.com | 05e7d19a6785768deef70d21ff4226e | NULL | NULL |
PHP web services are used to authenticate users and:
- Allow hubs to push data into the database
- Allow desktop/mobile clients to pull data from the database
Data is currently pushed to the database via a script at http://79.125.20.47/dataStore/push.php
Hubs need to generate JSON including one or more power values, ID of the sensors generating the data, ID of the hub, a timestamp of the time at which the data was generated and the username/API key of a valid datastore user. An example of a valid JSON object is shown below:
{
"readingSet": {
"user": "user@user.com",
"apiKey": "123456789012345667890",
"timeStamp": "2011-06-15 13:22:04",
"hubId": 26,
"readings": [
{
"sensorId": 8,
"load": 100
}
]
}
}
A text representation of the JSON object is passed as a "json" parameter in an HTTP GET request to the php script, which parses the JSON, authenticates the user making the request, and places the data in the database.
It makes more sense for this to happen via HTTP POST or PUT; this will be changed at some point.
We have built a set of demonstration clients to show how the energy data might be visualised (e.g. http://79.125.20.47/bzb/graph.php - please enter "test@caware.com" as a username and "energy" as a password when prompted).
Currently, these clients are accessing the database using a set of PHP web services. We are beginning to define the API for these web services to allow access to the database in a more controlled manner.
Functions that are currently available in the API are:
- currentNetData - returns information about all current sensors and hubs that appear to be active (have sent data to the database in the last 2 hours), and the most recent energy data for each sensor
- currentNetLoad - returns the current total load for all hubs that appear to be active (sum of load for all sensors that appear to be active in the database)
- currentHubData - returns information about all the active sensors attached to a particular hub, including the most recent energy data for each sensor
- currentHubLoad - returns the current total load for a particular hub (sum of all active sensors on the hub)
- prev2 - returns the current total load for all active hubs, the number of active hubs, and the mean load for these hubs for this time of day
- gridData - returns data about the UK National Grid (whether electricity frequency is high or low, and whether carbon emissions are high or low)
To view working examples of the above functions, use the following URL: http://79.125.20.47/dataStore/examples.php?u=test@caware.com&p=energy .
We are using a system of energy meters originally supplied by the University. These are rather more expensive than we would want to use in the long-term, although we will be adding more of a similar type in the next month.
The meters are meant for heavy installation and can give many more parameters than we would likely want. The types are summarised below.
-
Autometers IC970C
-
UK price: £150
-
Phases: 3
-
Harmonics: No
-
Communication: Modbus/RS485
-
-
Autometers IC995
-
UK price: £250
-
Phases: 3
-
Harmonics: Yes
-
Communication: Modbus/RS485
-
We will have approximately 50 meters in total. Only a few are attached to multiple phases, so the total number of circuits monitored will be around 60. We aim to provide half hour energy time series (available on the web) for all meters together with descriptive information. Information will be available for server rooms, lighting on each “corridor”, the collective consumption of sockets on a “corridor”, the various air conditioning units, etc.)
The meters attach via a Modbus (logical) over RS485 (physical) bus. This design dates from the mid 1970’s. One can obtain USB/RS485 interfaces cheaply and we poll the meters from an application running on Linux.
The Modbus/RS485 is a popular choice for heavy plant monitoring. We have also found meters which connect to Ethernet and run Modbus over TCP. (Isn’t backwards compatibility marvelous ☹.)
In the future we aim to use less expensive sensors to provide similar information at finer granularity.