This repository accompanies Goldfeder, J., Sipple, J., Real-World Data and Calibrated Simulation Suite for Offline Training of Reinforcement Learning Agents to Optimize Energy and Emission in Office Buildings, currently under review at Neurips 2024, and builds off of Goldfeder, J., Sipple, J., (2023). A Lightweight Calibrated Simulation Enabling Efficient Offline Learning for Optimal Control of Real Buildings, BuildSys ’23, November 15–16, 2023, Istanbul, Turkey
The best place to jump in is the Soft Actor Critic Demo notebook, available in notebooks/SAC_Demo.ipynb
This will walk you through:
-
Creating an RL (gym compatible) envronment
-
Visualizing the env
-
Training an agent using the Tensorflow Agents Library
Before you run this notebook, make sure to go through the setup instructions below to ensure the notebook runs successfully.
Follow these steps to setup locally before you run the notebooks/SAC_Demo.ipynb notebook. Note: this will only work on linux, as some libraries are not supported by other operating systems.
-
Clone the repository
-
Ensure you have
protocandffmpeginstalled, as well aspython >=3.10.12 and <3.12. You can install these runningsudo apt install -y protobuf-compilerandsudo apt install -y ffmpeg -
Create a virtual environment by running
python -m venv .venv. Then, install poetry withpip install poetry -
Install the dependencies by running
poetry install -
Build the
.protofiles atsmart_control/protointo python files by runningcd smart_control/proto && protoc --python_out=. smart_control_building.proto smart_control_normalization.proto smart_control_reward.proto && cd ../.. -
Modify the value of
VIDEO_PATH_ROOTatsmart_control/simulator/constants.py. This is the path where simulation videos will be stored -
Now in the
notebooks/SAC_Demo.ipynbnotebook, modify the values ofdata_path,metrics_path,output_data_pathandroot_dir. In particular,data_pathshould point to thesim_config.ginfile atsmart_control/configs/sim_config.gin -
Now you are ready to run the
notebooks/SAC_Demo.ipynbnotebook
In addition to our calibrated simulator, we released 6 years of data on 3 buildings, for further calibration, and to use, in conjunction with the simulator, for training and evaluating RL models. The dataset is part of Tensorflow Datasets