A full-featured transparent data preparation routine from raw data to POMMES model inputs
This is the data preparation routine of the fundamental power market model POMMES (POwer Market Model of Energy and reSources).
Please navigate to the section of interest to find out more.
POMMES itself is a cosmos consisting of a dispatch model, a data preparation routine (stored in this repository and described here) and an investment model for the German wholesale power market. The model was originally developed by a group of researchers and students at the chair of Energy and Resources Management of TU Berlin and is now maintained by a group of alumni and open for other contributions.
If you are interested in the actual dispatch or investment model, please find more information here:
- pommesdispatch: A bottom-up fundamental power market model for the German electricity sector
- pommesinvest: A multi-period integrated investment and dispatch model for the German power sector (upcoming).
The data preparation is mainly carried out in this jupyter notebook. The data sources used as well as the calculation and transformation steps applied are described in a transparent manner. In addition to that, there is a documentation of pommesdata on readthedocs. This in turn contains a documentation of the functions and classes used for data preparation.
There are two use cases for using pommesdata:
- Using readily prepared output data sets as
pommesdispatchorpommesinvestinputs - Understanding and manipulating the data prep process (inspecting / developing)
If you are only interested in the readily prepared data sets (option 1), you can obtain them from zenodo and download it here: https://zenodo.org/
If you are interested in understanding the data preparation process itself or if you wish to include own additions, changes or assumptions, you can fork and then clone the repository, in order to copy the files locally by typing
git clone https://github.com/pommes-public/pommesdata.git
After cloning the repository, you have to install the required dependencies. Make sure you have conda installed as a package manager. If not, you can download it here. Open a command shell and navigate to the folder where you copied the environment to. Use the following command to install dependencies
conda env create -f environment.yml
Activate your environment by typing
conda activate pommes_data
Every kind of contribution or feedback is warmly welcome.
We use the GitHub issue management as well as pull requests for collaboration.
We try to stick to the PEP8 coding standards.
The jupyter notebook for the data preparation does not (necessarily have to) meet PEP8 standards, though readability should be made sure.
The following people have contributed in the following manner to pommesdata:
| Name | Contribution | Status |
|---|---|---|
| Yannick Werner | major development & conceptualization conceptualization, main data preparation routines (status quo data for all components, detailed RES, interconnector and hydro data), architecture |
coordinator & maintainer, developer & corresponding author |
| Johannes Kochems | major development & conceptualization conceptualization, co-development of main data preparation routines (esp. future projection for all components, RES tender data and LCOE estimates, documentation), architecture publishing process |
coordinator & co-maintainer, developer & corresponding author |
| Leticia Encinas Rosa | development early-stage contributions to conventional power plant data collection for Germany (technical data, data processing routines) |
former developer (research associate) |
| Carla Spiller | development early-stage contributions to conventional power plant data collection for Germany (technical data, data processing routines) |
former developer (student assistant) |
| Sophie Westphal | development contributions to cost data collection for conventional plants (data bundling and processing routines) |
former developer (student assistant) |
| Julian Endres | development early-stage contributions to conventional power plant data collection for Germany (location, technical data) |
former developer (student assistant) |
| Julien Faist | development contributions to conventional power plant data collection for Germany (shutdowns, new constructions) |
former developer (master's student) |
| Timona Ghosh | development early-stage development of interconnector exchange (approach and data) |
former developer (master's student) |
| Johannes Giehl | development conceptualization and data licensing information |
developer |
| Christian Fraatz | development early-stage contributions to European conventional power plant data (location and data processing) |
former developer (bachelor's student) |
| Robin Claus | development early-stage contributions to German conventional power plant data (efficiencies) |
former developer (student assistant) |
| Daniel Peschel | development early-stage input data contribution to German conventional power plant data (CHP information) |
former developer (master's student) |
| Conrad Nicklisch | development early-stage contribution to RES cost information |
former developer (bachelor's student) |
| Benjamin Grosse | development early-stage contributions to conventional power plants data & support |
developer |
| Joachim Müller-Kirchenbauer | support & conceptualization early-stage conceptualization, funding |
supporter (university professor) |
Note: Not every single contribution is reflected in the current version of
pommesdata. This is especially true for those marked as early-stage
contributions that may have been extended, altered or sometimes discarded.
Nonetheless, all people listed have made valuable contributions. The ones
discarded might be re-integrated at some point in time.
Dedicated contributions to pommesdispatch and pommesinvest are not included
in the list, but listed individually for these projects.
A publication using and introducing pommesdispatch is currently in preparation.
This in turn will make use of data collected with pommesdata.
Data sets created with pommesdata are shared at zenodo.
If you use these, please refer to the citation information given at zenodo.
If you are using pommesdata for your own analyses, we recommend citing as:
Werner, Y.; Kochems, J. et al. (2021): pommesdata. A full-featured transparent data preparation routine from raw data to POMMES model inputs. https://github.com/pommes-public/pommesdata, accessed YYYY-MM-DD.
We furthermore recommend naming the version tag or the commit hash used for the sake of transparency and reproducibility.
Also see CITATION.cff for citation information. Licensing information stated in the CITATION.cff is only applicable for the code itself, see license.
Licensing for the code - in the following referred to as software - and the input data used differs. For the licensing of the data, please see the detailed list of data sets below.
Copyright 2021 pommes developer group
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
The following table contains the primary data sources used to create data sets used for POMMES models. The licensing of the different sources differs and the table should provide an overview over the licences used. Thus, we cannot publish all the data under an open license, such as a Creative Commons Attribution license. Please be aware that some data might be subject to copyright.
The data is provided with no license. Please refer to the above licensing information.