Skip to content
Branch: master
Find file History
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
..
Failed to load latest commit information.
figs
__init__.py
cleaning_data.py
data_scraping.py
make_training_dataset.py
readme.md
training.py
visualization.ipynb

readme.md

elexton_data_scraping.py

This script pulls data from ELEXON using their API. It requires getting an API key which you can get here.

Your API key is passed into the script from the command line

$ python elexon_data_scraping.py --key YOURKEYHERE

The basic flow of the script is

  • form a URL to query the ELEXON API for a single SettlementDate
  • the requests library is used to parse the response from the API, searching for the columns specified
  • a pandas DataFrame is created from the parsed XML, with the timestamp converted from London time to GMT. This is to avoid the complication of daylight savings time

The script is setup to iterate over multiple days and to save all data to a csv. Saving a copy of the raw data is good practice - cleaning and processing of the ELEXON data is done in make_dataset.py

elexon_clean_data.py

$ python clean_data.py 

Bespoke data cleaning for the Elexon data. Care is taken to remove duplicates and fill missing data.

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.