Introducing OpenStreetMap high-voltage grid to PyPSA-Eur

[bobbyxng]

closes #1100

Changes proposed in this Pull Request

This pull request introduces a first functioning implementation of the high voltage-electricity grid (incl. 200 kV and above) extracted from OpenStreetMap (OSM, https://www.openstreetmap.org). The implementation is proposed in such a way that the user can freely determine whether to use the existing electricity network based on the GridKit extract of the ENTSO-E online map or based on OSM.

Components/scripts

• retrieve_osm_data.py: Retrieves OSM data for the specified country using the overpass API and save it
  to the specified output (json) files. Note that overpass requests are based on a fair use policy. retrieve_osm_data is meant to be used in a way that respects this policy by fetching the needed data once, only.
• clean_osm_data.py: The script performs various cleaning operations on the OSM data, including:
  • Cleaning voltage, circuits, cables, wires, and frequency columns
  • Splitting semicolon-separated cells into new rows
  • Determining missing parameters such as number of circuits heuristically, if such information is missing.
  • Adding line endings to substations based on line data
• build_osm_network.py: Modified implementation of PyPSA-Earth's build_osm_network.py script, which creates a connected network, by
  • aggregating close buses and setting bus ids
  • connecting cleaned lines to buses, respectively
  • and finally creating PyPSA-Eur friendly input files (.csv / .geojson for quick debugging and visualisation) in order to create the base network base.nc

Checklist

• I tested my contribution locally and it seems to work fine.
• Code and workflow changes are sufficiently documented.
• Changed dependencies are added to envs/environment.yaml.
• Changes in configuration options are added in all of config.default.yaml.
• Changes in configuration options are also documented in doc/configtables/*.csv.
• A release note doc/release_notes.rst is added.

Known issues and simplifications

• While DC links are also extracted from OSM, their power rating is unfortunately not consistently included in the raw data. As such, p_nom is currently calculated using the extracted voltage and the two DC line types available in PyPSA. In the future, we will determine the correct p_nom, e.g. by filling in the missing data to OSM, directly.
• Currently, two implementations of base_network.py (and base_network_osm.py, respectively) exist. This is partly related to how links are currently being handled in the OSM implementation. In the near future, this will be consolidated, so only one base_network.py for both GridKit and OSM-sourced electricity networks is needed.
• TYNDP and NEP projects are not implemented yet. This feature is under development.

Benchmarks and validation

Benchmarks (exemplary for Europe retion)

• retrieve_osm_data.py: 03:01 min (100 Mbit/s connection)
• clean_osm_data.py: 01:04 min
• build_osm_network.py: 02:08 min
• base_network.py: 00:18 min

Comparison of results/validation

Comparing route and circuit lengths against official ENTSO-E data (Inventory of transmission, https://www.entsoe.eu/data/power-stats/)

Base network overview

Line volume per NUTS1 region

• RMSE (PyPSA-Earth): $1.156 * 10^6$
• RMSE (PyPSA-Eur): $0.834 * 10^6$

Cross-regional capacity between NUTS1 regions

![scatterplot_crossregion_capacity](https://github.com/PyPSA/pypsa-eur/assets/36541459/cc8b5dc6-06c2-47a3-81

23-6d9daa0810ae)

• RMSE (PyPSA-Earth): $5.659* 10^3$
• RMSE (PyPSA-Eur): $3.448 * 10^3$

Delta in electricity generation per country

Transmission (25h resolution, full-year run, no TYNDP projects, no line expansion)