Codebase to reproduce the experiments performed in the paper (pre-print, under review):
- Brusaferri, A., Ballarino, A., Grossi, L., & Laurini, F. (2024). On-line conformalized neural networks ensembles for probabilistic forecasting of day-ahead electricity prices, https://arxiv.org/abs/2404.02722
The results of the experiments are stored as pickle files within the experiments/task folder,
aggregated by regions-> methods-> recalibration runs.
Note: in the code we used 'DE' to label the German market while in the paper we employed 'GE' since 'DE' was already employed to label Deep Ensembles.
The employed packages versions are stored in the requirements.txt file (Python 3.8.10). Besides, the code of DM-tests, Kupiec tests, Distributional NNs and Conformal PI are built upon https://github.com/jeslago/epftoolbox, https://github.com/rafa-rod/vartests, https://github.com/gmarcjasz/distributionalnn and https://github.com/aangelopoulos/conformal-time-series respectively.
The script results_analysis.py contains the functions to obtain the plots/tables in the Results section of the paper from the stored pickle.
To execute the recalibration experiments from scratch (i.e., models re-training), open the run_recalibration.py script,
select the dataset to execute in the PF_task_name variable and run the script.
Specific local minima reached by the training algorithm may lead to fluctuations in the test predictions (e.g., QR vs JSU vs Stu).
Still, the implemented Conformal Prediction based techniques are expected to improve the hourly calibration of the backbone models across the different settings.
The run_recalibration.py script will store the experiments results in the related results subfolders.
Create a copy of the recalib_opt_grid_1_(1-4) folders by defining a new name (e.g., my_recalib_opt_grid_1_(1-4)) before running the experiments
to keep the original experiments results, otherwise they will be updated during each run of the script.
Keep the variable hyper_mode set to 'load_tuned' to load the stored hyperparameters values.
Set it to 'optuna_tuner' for executing also hyperparameter search from scratch.
Once the recalibration runs are completed, run the exec_qra_cp.py script
to execute the post-processing routines (i.e., Quantile Regression Averaging and Conformal Prediction).
If you created your own experimental copy, assign the chosen name to the run_id variable within both run_recalibration.py, exec_qra_cp.py and results_analysis.py
If you use this code in your publication, please cite our paper: https://arxiv.org/abs/2404.02722
@misc{brusaferri2024online,
title={On-line conformalized neural networks ensembles for probabilistic forecasting of day-ahead electricity prices},
author={Alessandro Brusaferri and Andrea Ballarino and Luigi Grossi and Fabrizio Laurini},
year={2024},
eprint={2404.02722},
archivePrefix={arXiv},
primaryClass={cs.LG}}