An important factor to guarantee a responsible use of data-driven systems is that we should be able to communicate their uncertainty to decision makers. This can be accomplished by constructing prediction sets, which provide an intuitive measure of the limits of predictive performance.
This package contains a Python implementation of Rolling Conformal Inference (Rolling CI) [1] methodology for constructing distribution-free prediction sets.
Rolling CI is a method that reliably reports the uncertainty of a target variable response in the time-series setting and provably attains the user-specified coverage level over long-time intervals.
[1] Shai Feldman, Stephen Bates, Yaniv Romano, “Conformalized Online Learning: Online Calibration Without a Calibration Set” 2022.
This package is self-contained and implemented in python.
Part of the code is a taken from the oqr and mqr packages.
- python
- numpy
- scipy
- scikit-learn
- pytorch
- pandas
The development version is available here on github:
git clone https://github.com/shai128/rci.gitComparisons to competitive methods and can be found in display_results.ipynb.
The code available under /reproducible_experiments/ in the repository replicates the experimental results in [1].
-
Power: Power consumption of Tetouan city Data Set.
-
Energy: Appliances energy prediction Data Set.
-
Traffic: Metro Interstate Traffic Volume Data Set.
-
Wind: Wind Power in Germany.
-
Power: French electricity prices [2].
[2] Margaux Zaffran, Aymeric Dieuleveut, Olivier Féron, Yannig Goude, Julie Josse, “Adaptive Conformal Predictions for Time Series.” 2022.