A full-featured transparent data preparation routine from raw data to POMMES model inputs
This is the data preparation routine of the fundamental power market model POMMES (POwer Market Model of Energy and reSources).
Please navigate to the section of interest to find out more.
POMMES itself is a cosmos consisting of a dispatch model, a data preparation routine (stored in this repository and described here) and an investment model for the German wholesale power market. The model was originally developed by a group of researchers and students at the chair of Energy and Resources Management of TU Berlin and is now maintained by a group of alumni and open for other contributions.
If you are interested in the actual dispatch or investment model, please find more information here:
- pommesdispatch: A bottom-up fundamental power market model for the German electricity sector
- pommesinvest: A multi-period integrated investment and dispatch model for the German power sector (upcoming).
The data preparation is mainly carried out in this jupyter notebook. The data sources used as well as the calculation and transformation steps applied are described in a transparent manner. In addition to that, there is a documentation of pommesdata on readthedocs. This in turn contains a documentation of the functions and classes used for data preparation.
There are two use cases for using pommesdata:
- Using readily prepared output data sets as
pommesdispatchorpommesinvestinputs - Understanding and manipulating the data prep process (inspecting / developing)
If you are only interested in the readily prepared data sets (option 1), you can obtain them from zenodo and download it here: https://zenodo.org/
If you are interested in understanding the data preparation process itself or if you wish to include own additions, changes or assumptions, you can fork and then clone the repository, in order to copy the files locally by typing
git clone https://github.com/pommes-public/pommesdata.git
After cloning the repository, you have to install the required dependencies. Make sure you have conda installed as a package manager. If not, you can download it here. Open a command shell and navigate to the folder where you copied the environment to. Use the following command to install dependencies
conda env create -f pommesdata_explicit.yml
Activate your environment by typing
conda activate pommesdata_explicit
Note: Dependencies have not been regularly updated. Thus, use the listed explicit
dependencies from pommesdata_explicit.yml for now and not the environment.yml file.
Every kind of contribution or feedback is warmly welcome.
We use the GitHub issue management as well as pull requests for collaboration.
We try to stick to the PEP8 coding standards.
The jupyter notebook for the data preparation does not (necessarily have to) meet PEP8 standards, though readability should be made sure.
- Authors of
pommesdataare Johannes Kochems and Yannick Werner. It is maintained by Johannes Kochems. - Florian Maurer contributed to the source code by providing a bug fix.
- All people mentioned below contributed to early-stage versions or predecessors of POMMES or ideally supported it.
The following people have contributed to POMMES. Most of these contributions belong to early-stage versions and are not part of the actual source code. Nonetheless, all contributions shall be acknowledged and the full list is provided for transparency reasons.
The main contributors are stated on top, the remainder is listed in alphabetical order.
| Name | Contribution |
|---|---|
| Johannes Kochems | major development & conceptualization conceptualization, development of all investment-related parts; development of main data preparation routines (esp. future projection for all components, RES tender data and LCOE estimates, documentation), architecture, publishing process, maintenance |
| Yannick Werner | major development & conceptualization conceptualization, development of main data preparation routines (status quo data for all components, detailed RES, interconnector and hydro data), architecture |
| Benjamin Grosse | data collection for conventional power plants in early development stage, ideal support and conceptionel counseling |
| Carla Spiller | data collection for conventional power plants in early stage development as an input to pommesdata; co-development of rolling horizon dispatch modelling in predecessor of pommesdispatch |
| Christian Fraatz | data collection for conventional power plants in early stage development as an input to pommesdata |
| Conrad Nicklisch | data collection for RES in early stage development as an input to pommesdata |
| Daniel Peschel | data collection on CHP power plants as an input to pommesdata |
| Dr. Johannes Giehl | conceptionel support and research of data licensing; conceptionel support for investment modelling in pommesinvest |
| Dr. Paul Verwiebe | development of small test models as a predecessor of POMMES |
| Fabian Büllesbach | development of a predecessor of the rolling horizon modeling approach in pommesdispatch |
| Flora von Mikulicz-Radecki | extensive code and functionality testing in an early development stage for predecessors of pommesdispatch and pommesinvest |
| Florian Maurer | support with / fix for python dependencies |
| Hannes Kachel | development and analysis of approaches for complexity reduction in a predecessor of pommesinvest |
| Julian Endres | data collection for costs and conventional power plants in early stage development |
| Julien Faist | data collection for original coal power plant shutdown and planned installation of new power plants for pommesdata; co-development of a predecessor of pommesinvest |
| Leticia Encinas Rosa | ata collection for conventional power plants in early stage development as an input to pommesdata |
| Prof. Dr.-Ing. Joachim Müller-Kirchenbauer | funding, enabling and conceptual support |
| Robin Claus | data collection for RES in early stage development as an input to pommesdata |
| Sophie Westphal | data collection for costs and conventional power plants in early stage development as an input for pommesdata |
| Timona Ghosh | data collection for interconnector data as an input to pommesdata |
Data sets created with pommesdata are shared at zenodo.
If you use these, please refer to the citation information given at zenodo.
If you are using pommesdata for your own analyses, we recommend citing as:
Kochems, J. & Wener, Y. (2024): pommesdata. A full-featured transparent data preparation routine from raw data to POMMES model inputs. https://github.com/pommes-public/pommesdata, accessed YYYY-MM-DD.
We furthermore recommend naming the version tag or the commit hash used for the sake of transparency and reproducibility.
Also see CITATION.cff for citation information. Licensing information stated in the CITATION.cff is only applicable for the code itself, see license.
Licensing for the code - in the following referred to as software - and the input data used differs. For the licensing of the data, please see the detailed list of data sets below.
Copyright 2024 pommes developer group
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
The following table contains the primary data sources used to create data sets used for POMMES models. The licensing of the different sources differs and the table should provide an overview over the licences used. Thus, we cannot publish all the data under an open license, such as a Creative Commons Attribution license. Please be aware that some data might be subject to copyright.
The data is provided with no license. Please refer to the above licensing information.