New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simulator of disaggregated domestic electricity demand at 1 Hz #26

Open
JackKelly opened this Issue Dec 6, 2013 · 20 comments

Comments

Projects
None yet
5 participants
@JackKelly
Contributor

JackKelly commented Dec 6, 2013

We're not planning to build the simulator for a while but I wanted to start this feature request so we can keep notes about the simulator.

Some specific feature ideas:

  • Test the effect of:
    • change sample rate
    • multiple appliance on at same time
    • different types of appliance, etc
  • Generating scenarios:
    • takes a simple text script as its input (purely deterministic) (Text scripts would allow researchers to share their “scenarios”.)
    • nice GUI for creating these text scripts
    • sample randomly from the probability distributions of time-of-use of each appliance that we calculate from real datasets, so it can automatically create an endless stream of realistic simulations

References

If we want to go down the route of trying to simulate appliances themselves (rather than stitching together appliance waveforms) then Dr Mark Bilton's PhD thesis might be of interest... he built a simulator for appliances down to a remarkable level of detail. Appliances are modelled pretty much from first principals. Appliances are described in terms of a FSM, volumes of physical containers (to simulate heating and cooling), U-values of insulators for fridges, etc etc. Very very detailed stuff. http://www.mendeley.com/c/6313531404/g/3295941/bilton-2011-electricity-demand-measurement-modelling-and-management-of-uk-homes/

And here's the Barker et al paper that Nipun mentioned: "Empirical Characterization and Modeling of Electrical Loads in Smart Homes". http://lass.cs.umass.edu/papers/pdf/igcc13-model.pdf

@nipunbatra

This comment has been minimized.

Show comment
Hide comment
@nipunbatra

nipunbatra Dec 12, 2013

Member

Before I forget: Maybe at some later point of time, we would like to have a look at Energy+ Simulator

The following paper from Buildsys 2012, which won the best demo award might be of interest; it provides a Matlab and Simulink interface to Energy+

Member

nipunbatra commented Dec 12, 2013

Before I forget: Maybe at some later point of time, we would like to have a look at Energy+ Simulator

The following paper from Buildsys 2012, which won the best demo award might be of interest; it provides a Matlab and Simulink interface to Energy+

@nipunbatra

This comment has been minimized.

Show comment
Hide comment
@jkeirstead

This comment has been minimized.

Show comment
Hide comment
@jkeirstead

jkeirstead May 21, 2014

Loughborough University developed a domestic load simulator in VBA a few years ago, and I've ported it to Java and gussied it up a bit. The project's called SimElec and it will do appliance and bulb level simulation, with a UI for standard runs and a few more features hidden in the API for advanced users.

jkeirstead commented May 21, 2014

Loughborough University developed a domestic load simulator in VBA a few years ago, and I've ported it to Java and gussied it up a bit. The project's called SimElec and it will do appliance and bulb level simulation, with a UI for standard runs and a few more features hidden in the API for advanced users.

@JackKelly

This comment has been minimized.

Show comment
Hide comment
@JackKelly

JackKelly May 21, 2014

Contributor

That's awesome, thanks loads @jkeirstead ;)

Contributor

JackKelly commented May 21, 2014

That's awesome, thanks loads @jkeirstead ;)

@jkeirstead

This comment has been minimized.

Show comment
Hide comment
@jkeirstead

jkeirstead May 21, 2014

@nipunreddevil Oh, and while I think of it, I also have a project to connect Java to EnergyPlus, via jEPlus. It's unimaginatively called jE++.

jkeirstead commented May 21, 2014

@nipunreddevil Oh, and while I think of it, I also have a project to connect Java to EnergyPlus, via jEPlus. It's unimaginatively called jE++.

@nipunbatra

This comment has been minimized.

Show comment
Hide comment
@nipunbatra

nipunbatra May 21, 2014

Member

@jkeirstead Interesting! Would try to tinker around some time.

Member

nipunbatra commented May 21, 2014

@jkeirstead Interesting! Would try to tinker around some time.

@nipunbatra

This comment has been minimized.

Show comment
Hide comment
@nipunbatra

nipunbatra May 21, 2014

Member

@jkeirstead Do these repos have any accompanying article/paper/report yet?

Member

nipunbatra commented May 21, 2014

@jkeirstead Do these repos have any accompanying article/paper/report yet?

@jkeirstead

This comment has been minimized.

Show comment
Hide comment
@jkeirstead

jkeirstead May 21, 2014

@nipunreddevil For SimElec, there are three papers listed in the README and for jE++, it's basically a utility, so you'd probably want to cite either EnergyPlus for the underlying model or Yi Zhang's publications for parametetric simulations with jEPlus.

I've been using these tools to build a fancier version of this paper; conference presentation will be in two weeks and then we'll see about the paper after that.

jkeirstead commented May 21, 2014

@nipunreddevil For SimElec, there are three papers listed in the README and for jE++, it's basically a utility, so you'd probably want to cite either EnergyPlus for the underlying model or Yi Zhang's publications for parametetric simulations with jEPlus.

I've been using these tools to build a fancier version of this paper; conference presentation will be in two weeks and then we'll see about the paper after that.

@JackKelly

This comment has been minimized.

Show comment
Hide comment
@JackKelly

JackKelly Oct 5, 2016

Contributor

Has anyone come across a domestic electricity simulator which can simulate power demand at one-second intervals?

If not then I'm toying with the idea of building something. It might work something like this:

First, build simple statistical models of when appliances are used. These models would be trained from long-duration but low resolution datasets like HES and Dataport. The models would try to capture things like the time of day that appliances are used (e.g. toasters are often used in the morning); the correlation between appliances (e.g. toasters and kettles are often used within a few minutes of each other), the correlation between external temperature and appliances, etc.

You'd build different statistical models for each country. And, for each country, it would probably be necessary to create several models, for several different usage patterns (e.g. homes with young children; single-occupancy homes where the occupant is away during the week days etc.)

Then the system would extract the 1Hz power demand from real appliances from datasets such as Tracebase. These time series would be split up into single runs of each appliance.

(There are now 20 disaggregated datasets listed on the NILM wiki)

To simulate power data at 1Hz, the system would then sample randomly from the statistical model of appliance usage, conditional on real historical temperature data (from a time period which was not in the original data). Each artificial "house" would be made up of a collection of appliances, taken from the high frequency datasets.

To get even more sophisticated, you could imagine using something like Energy+ to model the building physics in order to model the internal temperature of the home.

Any thoughts?

Contributor

JackKelly commented Oct 5, 2016

Has anyone come across a domestic electricity simulator which can simulate power demand at one-second intervals?

If not then I'm toying with the idea of building something. It might work something like this:

First, build simple statistical models of when appliances are used. These models would be trained from long-duration but low resolution datasets like HES and Dataport. The models would try to capture things like the time of day that appliances are used (e.g. toasters are often used in the morning); the correlation between appliances (e.g. toasters and kettles are often used within a few minutes of each other), the correlation between external temperature and appliances, etc.

You'd build different statistical models for each country. And, for each country, it would probably be necessary to create several models, for several different usage patterns (e.g. homes with young children; single-occupancy homes where the occupant is away during the week days etc.)

Then the system would extract the 1Hz power demand from real appliances from datasets such as Tracebase. These time series would be split up into single runs of each appliance.

(There are now 20 disaggregated datasets listed on the NILM wiki)

To simulate power data at 1Hz, the system would then sample randomly from the statistical model of appliance usage, conditional on real historical temperature data (from a time period which was not in the original data). Each artificial "house" would be made up of a collection of appliances, taken from the high frequency datasets.

To get even more sophisticated, you could imagine using something like Energy+ to model the building physics in order to model the internal temperature of the home.

Any thoughts?

@oliparson

This comment has been minimized.

Show comment
Hide comment
@oliparson

oliparson Oct 5, 2016

Contributor

I remember reading a few papers that either focus on simulating households or evaluate their algorithms using simulated data. It might be worth carrying out a quick literature search first to check if there are any ideas you want to build on, e.g. http://dl.acm.org/citation.cfm?id=2559630

I doubt you'll find open source code for the whole solution though.

Also, I think NILM-Metadata project should come in handy here :)

Contributor

oliparson commented Oct 5, 2016

I remember reading a few papers that either focus on simulating households or evaluate their algorithms using simulated data. It might be worth carrying out a quick literature search first to check if there are any ideas you want to build on, e.g. http://dl.acm.org/citation.cfm?id=2559630

I doubt you'll find open source code for the whole solution though.

Also, I think NILM-Metadata project should come in handy here :)

@JackKelly

This comment has been minimized.

Show comment
Hide comment
@JackKelly

JackKelly Oct 5, 2016

Contributor

It might be worth carrying out a quick literature search first

yeah. I did a quick text search for "simulat" in my ref database and found 74 results :) (not all relevant)

Contributor

JackKelly commented Oct 5, 2016

It might be worth carrying out a quick literature search first

yeah. I did a quick text search for "simulat" in my ref database and found 74 results :) (not all relevant)

@marioberges

This comment has been minimized.

Show comment
Hide comment
@marioberges

marioberges commented Oct 8, 2016

How about Gridlab-D?
http://www.gridlabd.org/

@JackKelly

This comment has been minimized.

Show comment
Hide comment
@JackKelly

JackKelly Oct 8, 2016

Contributor

Thanks Mario, I'll look into Gridlab-D in more detail. It does say it can do sub-second resolution, which is promising. I wonder if it provides household-level power demand, though? And how realistic the appliance power signatures are. I'm just thinking out loud... I'll look into these questions myself if I do set out to build a simulator...

Contributor

JackKelly commented Oct 8, 2016

Thanks Mario, I'll look into Gridlab-D in more detail. It does say it can do sub-second resolution, which is promising. I wonder if it provides household-level power demand, though? And how realistic the appliance power signatures are. I'm just thinking out loud... I'll look into these questions myself if I do set out to build a simulator...

@JackKelly

This comment has been minimized.

Show comment
Hide comment
Contributor

JackKelly commented Oct 12, 2016

@JackKelly

This comment has been minimized.

Show comment
Hide comment
@JackKelly

JackKelly Oct 12, 2016

Contributor

A few other ideas for a simulator:

Two separate "modules":

  • One for generating "appliance activity scripts". This module would learn statistical models of appliance activity patterns from real data; and would learn different statistical models for different countries and different user groups; and would learn correlations between appliance usage and season and weather. To generate new "appliance activity scripts", the software would randomly sample from the statistical models of appliance behaviour. These "scripts" can then be used to drive real automated appliances (e.g. the labs set up by EPRI and PNNL). Or these same scripts could be used to drive computer-simulated power demand time series.
  • A second module would take two inputs: "appliance activity scripts" and real appliance-level power demand data. It would chop the appliance-level power demand data up into individual "activations" of each appliance and then simulate new household power demand by triggering these disembodied "activations" at the times described in the "appliance activity script".

One could imagine setting up a web service where people just download freshly simulated data, with whatever parameters they require.

Contributor

JackKelly commented Oct 12, 2016

A few other ideas for a simulator:

Two separate "modules":

  • One for generating "appliance activity scripts". This module would learn statistical models of appliance activity patterns from real data; and would learn different statistical models for different countries and different user groups; and would learn correlations between appliance usage and season and weather. To generate new "appliance activity scripts", the software would randomly sample from the statistical models of appliance behaviour. These "scripts" can then be used to drive real automated appliances (e.g. the labs set up by EPRI and PNNL). Or these same scripts could be used to drive computer-simulated power demand time series.
  • A second module would take two inputs: "appliance activity scripts" and real appliance-level power demand data. It would chop the appliance-level power demand data up into individual "activations" of each appliance and then simulate new household power demand by triggering these disembodied "activations" at the times described in the "appliance activity script".

One could imagine setting up a web service where people just download freshly simulated data, with whatever parameters they require.

@JackKelly

This comment has been minimized.

Show comment
Hide comment
@JackKelly

JackKelly Oct 12, 2016

Contributor

Would also be useful to be able to simulate "abnormal" (but physically plausible) behaviour. The sort of behaviour which does occur in real homes and which would provide an excellent opportunity to reduce energy consumption via feedback. e.g.:

  • a fridge with a broken door seal
  • stuff which is accidentally left on 24/7 and shouldn't be:
    • electric immersion heaters
    • electric swimming pool heater
    • electric food oven
    • a DVR which never switches off
  • an ancient fridge
Contributor

JackKelly commented Oct 12, 2016

Would also be useful to be able to simulate "abnormal" (but physically plausible) behaviour. The sort of behaviour which does occur in real homes and which would provide an excellent opportunity to reduce energy consumption via feedback. e.g.:

  • a fridge with a broken door seal
  • stuff which is accidentally left on 24/7 and shouldn't be:
    • electric immersion heaters
    • electric swimming pool heater
    • electric food oven
    • a DVR which never switches off
  • an ancient fridge

@JackKelly JackKelly changed the title from Appliance simulator to Simulator of disaggregated domestic electricity demand at 1 Hz Oct 12, 2016

@JackKelly

This comment has been minimized.

Show comment
Hide comment
@JackKelly

JackKelly Oct 18, 2016

Contributor

A few other quick notes on ideas which have come up during the NILM Workshop:

  • The simulator needs to be especially accurate at peak demand times. One use case for NILM is figuring out which appliances are contributing to peak demand.
  • We should put effort into metrics for measuring how "realistic" the simulated data is. Not entirely clear how to compute this!
  • A simple tool for visualising the data (and comparing the simulated data against real data) might be good.
  • Separate project: simple visualisation tool (possibly just an IPython project with sliders) for comparing NILM metrics and to see how the metric responds to certain "NILM errors" (an extension of PNNL's great work on systematically comparing metrics)
Contributor

JackKelly commented Oct 18, 2016

A few other quick notes on ideas which have come up during the NILM Workshop:

  • The simulator needs to be especially accurate at peak demand times. One use case for NILM is figuring out which appliances are contributing to peak demand.
  • We should put effort into metrics for measuring how "realistic" the simulated data is. Not entirely clear how to compute this!
  • A simple tool for visualising the data (and comparing the simulated data against real data) might be good.
  • Separate project: simple visualisation tool (possibly just an IPython project with sliders) for comparing NILM metrics and to see how the metric responds to certain "NILM errors" (an extension of PNNL's great work on systematically comparing metrics)
@JackKelly

This comment has been minimized.

Show comment
Hide comment
@JackKelly

JackKelly Oct 20, 2016

Contributor

I've started sketching out some very rough design ideas for the simulator on this Google doc. Comments welcome.

Contributor

JackKelly commented Oct 20, 2016

I've started sketching out some very rough design ideas for the simulator on this Google doc. Comments welcome.

@oliparson

This comment has been minimized.

Show comment
Hide comment
@JackKelly

This comment has been minimized.

Show comment
Hide comment
@JackKelly

JackKelly Oct 25, 2016

Contributor

Thanks Oli, I've added Dong Chen's paper to the list of references on the Google doc.

And here's the full reference:

Chen, D.; Irwin, D. & Shenoy, P. SmartSim: A Device-Accurate Smart Home Simulator for Energy Analytics. IEEE International Conference on Smart Grid Communications, 2016

Contributor

JackKelly commented Oct 25, 2016

Thanks Oli, I've added Dong Chen's paper to the list of references on the Google doc.

And here's the full reference:

Chen, D.; Irwin, D. & Shenoy, P. SmartSim: A Device-Accurate Smart Home Simulator for Energy Analytics. IEEE International Conference on Smart Grid Communications, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment