Reinforcement Learning Demo

From a joint effort of Sina Henning, Felix Krause and Johannes Spieß

Mandatory assignment 1 for Artificial Intelligence for Energy Informatics at University of Oslo (UiO) Autumn 2023.

Setup

Environment setup

Create new environment "venv"

python -m venv venv

Activate env in bash:

source venv/bin/activate

Install packages in venv

pip install -r requirements.txt

Add virtual environment to kernel

python -m ipykernel install --user --name=venv

Provide Datasets

For the demo, there is already all data needed in the data/directory.

For full replication (and more freedom regarding the energy demand), put the following datasets into a folder "/data" in this repo

• SolarIrradiance.csv
• rate_consumption_charge.csv/electricity price
• WindSpeed.csv

And put the "BASE" profiles in a folder "/data/residential_load_data_base"

• Residential load profiles: Check out data source here

Run training

Open the notebook "notebooks/RL_smartgrid_demo.ipynb" and follow instructions for a simple code demonstration.

For advanced replication with more options, open the notebook "notebooks/1_training.ipynb" and follow instructions. This needs the full datasets provided in the "data" folder.

Analysis of results

Open the notebook "notebooks/2_result_viz.ipynb".

Conventions

• The main unit used for energy is kWh
• The used time frame is an hour

Possible future work

In another iteration, one could try to improve the reward function, e.g.:

• penalizing switching energy sources on that are not optimal (i.e. solar on although there is no solar irradience, e.g. during night)
• including opportunity cost concept: negative reward proportional to "lost" money because a more expensive energy source was used than a cheap alternative (e.g. buying energy from grid although high solar irradience)