Merge pull request #99 from calliope-project/eez-from-api

Update EEZ data source to enable direct download in the workflow

CHANGELOG.md

@@ -1,6 +1,8 @@
 # Changelog
 
 ## 1.1 (unpublished)
+
+* **ADD** automatic download of EEZ (#99).
 * **ADD** Danish Energy Agency and Schroeder et al (2013) cost data as well as no hydro fixed costs as optional overrides (#18, #129).
 * **ADD** Add basic documentation to ReadTheDocs (#114).
 * **ADD** ability to move working directory (#45).
@@ -11,7 +13,8 @@
 * **ADD** sync infrastructure to easily send and receive files to and from a cluster (#74).
 * **ADD** optional email notifications whenever builds fail or succeed (#92).
 
-* **UPDATE** ENTSOE national electricity load data gap filling priority order included in config (#42)
+* **UPDATE** EEZ updated from v10 to v11 (difference in offshore area is < 1% for all relevant countries) (#99).
+* **UPDATE** ENTSOE national electricity load data gap filling priority order included in config (#42).
 * **UPDATE** IRENA hydro generation data from 2018 to 2020 (#40).
 * **UPDATE** Make scaling pumped hydro capacity according to Geth et al. (2015) optional with a boolean config parameter `scale-phs-according-to-geth-et-al` which defaults to False (no scaling) (#49).
 * **UPDATE** JRC hydro database v4 -> v7 (#48). This entails one patch being removed:

README.md

@@ -27,10 +27,6 @@ conda activate euro-calliope
 
 4. You need an account at the Copernicus Climate Data Service and you need to create a `$HOME/.cdsapirc` file with your credentials, see their [How To](https://cds.climate.copernicus.eu/api-how-to) (you do not need to manually install the client).
 
-5. Further, you need all data files that cannot be retrieved automatically:
-
-* [Maritime Boundaries v10 -> World Exclusive Economic Zones v10](http://www.marineregions.org/downloads.php), to be placed in `./data/World_EEZ_v10_20180221`
-
 ## Build the model
 
 Because input data is large, the actual model including its data is not part of this repository. To use the model, you first need to build it from input data and scripts. Running the build step will build the model in the `./model` folder.

config/default.yaml

@@ -2,7 +2,7 @@
 data-sources:
     biofuel-potentials: data/biofuels/potentials/{feedstock}.csv
     biofuel-costs: data/biofuels/costs/{feedstock}.csv
-    eez: data/World_EEZ_v10_20180221/eez_v10.shp
+    eez: https://geo.vliz.be/geoserver/MarineRegions/wfs?service=WFS&version=1.0.0&request=GetFeature&typeNames=MarineRegions:eez&outputFormat=SHAPE-ZIP
     irena-generation: data/irena/hydro-generation-europe.csv
     national-phs-storage-capacities: data/pumped-hydro/storage-capacities-gwh.csv
     capacity-factors: https://zenodo.org/record/3899687/files/{filename}?download=1
@@ -11,7 +11,7 @@ data-sources:
     hydro-stations: https://zenodo.org/record/4289229/files/energy-modelling-toolkit/hydro-power-database-v7.zip?download=1
     load: https://data.open-power-system-data.org/time_series/2019-06-05/time_series_60min_stacked.csv
     nuts: https://ec.europa.eu/eurostat/cache/GISCO/geodatafiles/NUTS_2013_01M_SH.zip
-    potentials: https://zenodo.org/record/3533038/files/possibility-for-electricity-autarky.zip
+    potentials: https://zenodo.org/record/5112963/files/possibility-for-electricity-autarky.zip
 root-directory: .
 cluster-sync:
     url: euler.ethz.ch

rules/shapes.smk

@@ -107,17 +107,26 @@ rule units_without_shape:
     script: "../scripts/shapes/nogeo.py"
 
 
+rule download_eez:
+    message: "Download Exclusive Economic Zones as zip"
+    output: protected("data/automatic/eez.zip")
+    params: url = config["data-sources"]["eez"]
+    conda: "../envs/shell.yaml"
+    shell: "curl -sLo {output} '{params.url}'"
+
+
 rule eez:
     message: "Clip exclusive economic zones to study area."
-    input: config["data-sources"]["eez"]
+    input: rules.download_eez.output[0]
     output: "build/data/eez.geojson"
     params:
         bounds="{x_min},{y_min},{x_max},{y_max}".format(**config["scope"]["bounds"]),
-        countries=",".join(["'{}'".format(country) for country in config["scope"]["countries"]])
+        countries=",".join(["'{}'".format(country) for country in config["scope"]["countries"]]),
     conda: "../envs/geo.yaml"
+    shadow: "minimal"
     shell:
         """
-        fio cat --bbox {params.bounds} {input}\
-        | fio filter "f.properties.Territory1 in [{params.countries}]"\
+        fio cat --bbox {params.bounds} "zip://{input}"\
+        | fio filter "f.properties.territory1 in [{params.countries}]"\
         | fio collect > {output}
         """

scripts/capacityfactors_offshore.py

@@ -11,9 +11,11 @@
 def capacityfactors(path_to_eez, path_to_shared_coast, path_to_timeseries,
                     threshold, path_to_result, year=None):
     """Generate offshore capacityfactor time series for each location."""
-    eez = gpd.read_file(path_to_eez).set_index("id").to_crs(EPSG3035).geometry
+    eez = gpd.read_file(path_to_eez).set_index("mrgid").to_crs(EPSG3035).geometry
     shared_coast = pd.read_csv(path_to_shared_coast, index_col=0)
     shared_coast.index = shared_coast.index.map(lambda x: x.replace(".", "-"))
+    shared_coast.columns = shared_coast.columns.astype(int)
+
     ts = xr.open_dataset(path_to_timeseries)
     if year:
         ts = ts.sel(time=str(year))