GridKit is a power grid extraction toolkit
GridKit uses spatial and topological analysis to transform map objects from OpenStreetMap into a network model of the electric power system. It has been developed in the context of the SciGRID project at the Next Energy research institute, to investigate the possibility of 'heuristic' analysis to augment the route-based analysis used in SciGRID. This has been implemented as a series of scripts for the PostgreSQL database using the PostGIS spatial extensions.
- Careful interpretation of results (e.g. power lines and stations may have multiple voltages).
- Realistic information on generation and load in the system (electrical supply and demand).
- Reasonable, possibly location specific assumptions on the impedance of lines, depending on their internal structure. Unlike SciGRID, such assumptions have not yet been applied.
Of note, PyPSA implements several methods of network simplification, which is in many cases essential for ensuring that the power flow computations remain managable.
Data exports created at March 14, 2016 for North America and Europe
can be downloaded from
zenodo. These exports are licensed
under the Open Database License because they derive from OpenStreetMap
data. They follow the same file structure as SciGRID.
util/network.py contains a parser for these files.
- Python (2.7 or higher)
- PostgreSQL (9.4 or higher)
- PostGIS (2.1 or higher)
- osm2pgsql (0.88.1 or higher)
- Optionally psycopg2 (2.6 or higher)
- Optionally osm-c-tools
How to Use (the simple way)
Download a power extract from enipedia:
wget http://enipedia.tudelft.nl/OpenStreetMap/EuropePower.zip unzip EuropePower.zip
Ensure you have a user for postgresql with permissions to create databases and modify schemas. For example:
createuser -d gridkit
python gridkit.py EuropePower.osm
--pg option takes a series of key=value pairs, which are parsed
into database connection options. For example to connect to a host on
10.0.0.160 listening on port 9000:
python gridkit.py --pg host=10.0.0.160 port=9000 EuropePower.osm
gridkit-highvoltage-vertices.csv contains a CSV file with
all high-voltage stations, and
contains a CSV file with all high-voltage lines. You may use
--full-export to export all other lines, too.
How to Use (the hard way)
Extract the power information
osmconvert my_area.osm.pbf -o=my_area.o5m osmfilter my_area.o5m --keep='power=*' -o=my_area_power.o5m
osmconvert planet-latest.osm.pbf -B=my_area.poly -o=my_area.o5m
GridKit assumes that you have the
available. Configuring access to the postgresql server is implemented
environment variables. Thus,
prior to importing the openstreetmap source data, you should use something like:
export PGUSER=my_username PGDATABASE=my_database PGPASSWORD=my_password
export PGHOST=server_hostname PGPORT=server_port
For more information, check out the linked documentation.
Import data using the script:
./import-data.sh /path/to/data.o5m database_name
database_name parameter is optional if the
environment variable has been set. Running the extraction process is just:
You should expect this, depending on your machine and the size of your dataset, to take anywhere from 5 minutes to a few hours. Afterwards, you can extract a copy of the network using:
psql -f export-topology.sql
Which will copy the network to a set of CSV files in
/tmp/heuristic_links.csvcontain the complete network at all voltage and frequency levels.
/tmp/heuristic_links_highvoltagecontain the high-voltage network (>220kV) at any frequency which is not 16.7Hz.
Some things to watch out for
This process requires a lot of memory and significant amounts of CPU time. It also makes a lot of copies of the same data. This is very useful for investigating issues and tracking changes by the system. It also means that you should probably not use this on resource-constrained systems, such as docker containers in constrained virtual machines or raspberry pi's.
Queries have been typically optimized to make the best possible use of indices, but whether they are actually used is sensitive to query planner specifics. Depending on the specifics, PostgreSQL tuning may be necessary to run the extraction process efficiently.
The resultant network will in almost all cases be considerably more complex than equivalent networks (e.g. from SciGRID or the Bialek model. For practical applications, it is highly advisable to use simplification.
Aside from the main codebase, some utilities have been implemented to enable analysis of the results. Notably:
util/network.pyallows for some simple analysis of the network, transformation into a PYPOWER powercase dictionary, and 'patching' of the network to propagate voltage and frequency information from neighbors.
util/load_polyfile.pytransforms a set of
polyfiles into import statements for PostgreSQL, to allow data statistics per area, among other things.