Skip to content

DrafProject/elmada

Repository files navigation

elmada logo


elmada: Dynamic electricity carbon emission factors and prices for Europe

Status: PyPI CI CI with conda codecov

Usage: python License: LGPL v3 status Downloads

Contribution: Imports: isort Code style: black Gitter Contributor Covenant

The open-source Python package elmada provides electricity carbon emission factors and wholesale prices for European countries. The target group includes modelers of distributed energy hubs who need electricity market data (short: elmada), e.g., to evaluate the environmental effect of demand response. elmada is part of the Draf Project but can be used as a standalone package.

Elmada scheme scribble

Features

  • Dynamic electricity Carbon Emission Factors (CEFs) are calculated depending on country and year in up to quarter-hourly resolution. There are two types of CEFs: Grid Mix Emission Factors (XEFs) and Marginal Emission Factors (MEFs). While XEFs reflect the carbon footprint of an electricity use (attributional approach), MEFs estimate the carbon impact (consequential approach) of a change in electricity demand (Learn more in the white paper from Tomorrow and WattTime). Choose between

    • XEFs from fuel type-specific ENTSO-E electricity generation data only for Germany (XEF_EP),
    • and XEFs & MEFs from merit order based simulations for 30 European Countries (XEF_PP, XEF_PWL, MEF_PP, MEF_PWL). The according Power Plant method (PP) and Piecewise Linear method (PWL) are described in the open-access Applied Energy paper. The data used depend on the method chosen, see scheme below.
  • Wholesale electricity prices are provided for European countries. You can choose between the real historical ENTSO-E data (hist_EP) or the simulation results of the PP / PWL method.

  • Other interesting market data such as merit order lists & plots, fuel-specific generation data, or power plant lists are provided as a by-product of the CEF calculations.

Methodology

With the XEF_EP method, XEFs are calculated by multiplying the share matrix S (fuel type specific share of electricity generation per time step from ENTSO-E) with the intensity vector ε (fuel type specific life cycle carbon emission intensities from Tranberg.2019):

The methods PP, PWL, and PWLv are explained in the Applied Energy paper. Here is an overview:

scheme_CEF_calculation

Data

Geographic scope

In elmada, two-letter country codes (ISO 3166-1 alpha-2) are used.

The countries supported by elmada can be seen in the map below which is the output of elmada.plots.cef_country_map(year=2020, method="XEF_EP").

cef_country_map

In the Usage section they are referred to as Europe30. They include:

  • 20 countries analyzed in the Applied Energy paper: AT, BE, CZ, DE, DK, ES, FI, FR, GB, GR, HU, IE, IT, LT, NL, PL, PT, RO, RS, SI
  • 8 countries with only one reported fossil fuel type: BA, CH, EE, LV, ME, MK, NO, SE
  • 2 countries where installed generation capacity data for 2019 were only available after the publication of the Applied Energy paper: BG, SK

Data modes

You can use elmada in two data modes which can be set with elmada.set_mode(mode=<MODE>):

  • mode="safe" (default):
    • Pre-cached data for 4 years and 20 countries are used. The data are described in the Applied Energy paper.
    • The years are 2017 to 2020 and the countries AT, BE, CZ, DE, DK, ES, FI, FR, GB, GR, HU, IE, IT, LT, NL, PL, PT, RO, RS, SI.
    • The data is available in the space-saving and quick-to-read Parquet format under .../safe_cache.
  • mode="live":
    • Up-to-date data are retrieved on demand and are cached to an OS-specific directory, see elmada.paths.CACHE_DIR. A symbolic link to it can be conveniently created by executing elmada.make_symlink_to_cache().
    • Available years are 2017 until the present.
    • Slow due to API requests.
    • Requires valid API keys of ENTSO-E, Morph, Quandl, see table below.

Data sources

Description Local data location Source Channel Involved in
Generation time series & installed generation capacities .../safe_cache or CACHE_DIR ENTSO-E 🔌 on-demand-retrieval via EntsoePandasClient (requires valid ENTSO-E API key) CEFs via EP, PP, PWL, PWLv
Carbon prices (EUA) .../safe_cache or CACHE_DIR Sandbag & ICE 🔌 on-demand-retrieval via Quandl (requires valid Quandl API key) CEFs via PP, PWL, PWLv
Share of CCGT among gas power plants .../safe_cache or CACHE_DIR GEO 🔌 on-demand-download via Morph (requires valid Morph API key) CEFs via PWL, PWLv
(Average) fossil power plants sizes .../safe_cache or CACHE_DIR GEO 🔌 on-demand-scraping via BeautifulSoup4 CEFs via PWL, PWLv
German fossil power plant list with efficiencies .../safe_cache or CACHE_DIR OPSD 🔌 on-demand-download from here CEFs via PP, PWL, PWLv
Transmission & distribution losses .../worldbank Worldbank 💾 manual download from here CEFs via PP, PWL, PWLv
Fuel prices for 2015 (+ trends) .../from_other.py (+ .../destatis) Konstantin.2017 (+ DESTATIS) 🔢 hard-coded values (+ 💾 manual download from here) CEFs via PP, PWL, PWLv
Fuel type-specific carbon emission intensities .../from_other.py & .../tranberg Quaschning & Tranberg.2019 🔢 hard-coded values CEFs via EP, PP, PWL, PWLv

Time zones

The data is in local time since the Draf Project focuses on the modeling of individual local energy hubs. Standard time is used i.e. daylight saving time is ignored. Also see this table of the time zones used.

Installation

Using pip

python -m pip install elmada

NOTE: Read here why you should use python -m pip instead of pip.

From source using conda

For a conda environment including a full editable elmada version do the following steps.

Clone the source repository:

git clone https://github.com/DrafProject/elmada.git
cd elmada

Create an conda environment based on environment.yml and install an editable local elmada version:

conda env create

Activate the environment:

conda activate elmada

From source without using conda

For Unix

git clone https://github.com/DrafProject/elmada.git
cd elmada
python3 -m venv env
source env/bin/activate
python -m pip install -e .[dev]

For Windows

git clone https://github.com/DrafProject/elmada.git
cd elmada
py -m venv env
.\env\Scripts\activate
py -m pip install -e .[dev]

Tests

This should always work:

pytest -m="not apikey"

This works only if API keys are set as described below:

pytest

Usage

import elmada

OPTIONAL: Set your API keys and go live mode

elmada.set_api_keys(entsoe="YOUR_ENTSOE_KEY", morph="YOUR_MORPH_KEY", quandl="YOUR_QUANDL_KEY")
# NOTE: API keys are stored in an OS-dependent config directory for later use.

elmada.set_mode("live")

Carbon Emission factors

elmada.get_emissions(year=2019, country="DE", method="MEF_PWL", freq="60min", use_datetime=True)

... returns marginal emission factors calculated by the PWL method with hourly datetime index:

2019-01-01 00:00:00     990.103492
2019-01-01 01:00:00     959.758367
                          ...
2019-12-31 22:00:00    1064.122146
2019-12-31 23:00:00    1049.852079
Freq: 60T, Name: MEFs, Length: 8760, dtype: float64

The method argument of get_emissions() takes strings that consists of two parts seperated by an underscore. The first part is the type of emission factor: grid mix emission factors (XEF) or marginal emission factors (MEF). The second part determines the calculation method: power plant method (PP), piecewise linear method (PWL), or piecewise linear method in validation mode (PWLv).

The first part can be omitted (_PP, _PWL, _PWLv) to return a DataFrame that includes additional information.

elmada.get_emissions(year=2019, country="DE", method="_PWL")

... returns all output from the PWL method:

      residual_load  total_load marginal_fuel  efficiency  marginal_cost         MEFs        XEFs
0          21115.00    51609.75       lignite    0.378432      40.889230   990.103492  204.730151
1          18919.50    51154.50       lignite    0.390397      39.636039   959.758367  164.716687
...             ...         ...           ...         ...            ...          ...         ...
8758       27116.00    41652.00       lignite    0.352109      43.946047  1064.122146  388.542911
8759       25437.75    39262.75       lignite    0.356895      43.356723  1049.852079  376.009477
[8760 rows x 7 columns]

Additionally, XEFs can be calculated from historic fuel type-specific generation data (XEF_EP).

Here is an overview of valid method argument values:

method Return type Return values Restriction
XEF_PP Series XEFs using PP method DE
XEF_PWL Series XEFs using PWL method Europe30
XEF_PWLv Series XEFs using PWLv method DE
MEF_PP Series MEFs from PP method DE
MEF_PWL Series MEFs using PWL method Europe30
MEF_PWLv Series MEFs using PWLv method DE
_PP Dataframe extended data for PP method DE
_PWL Dataframe extended data for PWL method Europe30
_PWLv Dataframe extended data for PWLv method DE
XEF_EP Series XEFs using fuel type-specific generation data from ENTSO-E Europe30

You can plot the carbon emission factors with

elmada.plots.cefs_scatter(year=2019, country="DE", method="MEF_PP")

CEFs

Wholesale prices

elmada.get_prices(year=2019, country="DE", method="hist_EP")
0       28.32
1       10.07
        ...  
8758    38.88
8759    37.39
Length: 8760, dtype: float64

Possible values for the method argument of get_prices() are:

method Description Restriction
PP Using the power plant method DE
PWL Using piecewise linear method Europe30
PWLv Using piecewise linear method in validation mode DE
hist_EP Using historic ENTSO-E data Europe30 without BA, ME, MK
hist_SM Using historic Smard data used only as backup for DE, 2015 and 2018

Merit order

elmada.plots.merit_order(year=2019, country="DE", method="PP")

... plots the merit order:

merit_order

elmada.get_merit_order(year=2019, country="DE", method="PP")

... returns the merit order as DataFrame with detailed information on individual power plant blocks.

Pre-processed data

The following table describes additional elmada functions that provide pre-processed data. Keyword arguments are for example kw = dict(year=2019, freq="60min", country="DE").

elmada. function call Return type (Dimensions) Return value Usage in elmada Used within
get_el_national_generation(**kw) DataFrame (time, fuel type) National electricity generation Share matrix S XEF_EP method
get_el_national_generation(**kw).sum(axis=1) Series (time) Total national electricity generation Proxy for the total load XEFs calculations
get_residual_load(**kw) Series (time) Conventional national generation Proxy for the residual load (see scheme above) PP, PWL and PWLv

Contributing

Contributions in any form are welcome! To contribute changes, please have a look at our contributing guidelines.

In short:

  1. Fork the project and create a feature branch to work on in your fork (git checkout -b new-feature).
  2. Commit your changes to the feature branch and push the branch to GitHub (git push origin my-new-feature).
  3. On GitHub, create a new pull request from the feature branch.

Citing elmada

If you use elmada for academic work please cite this paper published in the Journal for Open Source Software:

status

@article{Fleschutz2021,
  title = {elmada: Dynamic electricity carbon emission factors and prices for Europe},
  author = {Markus Fleschutz and Michael D. Murphy},
  journal = {Journal of Open Source Software},
  publisher = {The Open Journal},
  year = {2021},
  volume = {6},
  number = {66},
  pages = {3625},
  doi = {10.21105/joss.03625},
}

If you use the PP or PWL method, please also cite the open-access Applied Energy paper:

APEN

@article{Fleschutz2021b,
  title = {The effect of price-based demand response on carbon emissions in European electricity markets: The importance of adequate carbon prices},
  author = {Markus Fleschutz and Markus Bohlayer and Marco Braun and Gregor Henze and Michael D. Murphy},
  journal = {Applied Energy},
  year = {2021},
  volume = {295},
  issn = {0306-2619},
  pages = {117040},
  doi = {10.1016/j.apenergy.2021.117040},
}

License

Copyright (c) 2021 Markus Fleschutz

License: LGPL v3

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.