[ENH] wishlist of interval and probabilistic foreasters

[fkiraly]

This issue is to create a wishlist of interval forecasters and probabilistic forecasters.
Anyone can add wishes here, in the end we prettify and move to the forecasting wishlist.

Interval forecasters have predict_quantile or predict_interval.
(or predict_var but that's more variance forecaster)

Probabilistic forecasters will have predict_proba, i.e., predict full distributions.

We should also include compositors, especially those converting point foreceasters into interval or probabilistic forecasters.

Inspiration can be the M5 competition, but we should also make sure we have "simple baselines" such as "constant variance = back-test aggregate variance" etc.

[ltsaprounis]

1. Amazon's Level Set Forecaster (LSF) - very promising results from their presentation in the M5 conference. In short they took the accuracy competition winning method, passed it through the LSF algorithm and they showed that they beat the winner of the uncertainty competition. This is already implemented in gluon-ts
2. Prediction Intervals from Bootstrapped residuals

[fkiraly]

From my side:

• NaiveForecaster should provide various strategies for estimating forecast variances: squared differences, smoothing residuals, etc. Few shape/scale distributions (normal, laplace). And, going forward, for variance/scale: constant, or "number time steps forward times one step forward".
• naive strategy: take a window over last n time points, forecast the empirical distribution corresponding to this.
• variant of this: instead of empirical distribution, use a kernel density estimator on top of it. May require skpro, can wrap a kernel density estimator.
• there should be a "squared residuals" (or "absolute residuals") wrapper, "squaring residuals" and related strategies are common compositors to add a volatility/uncertainty prediction. This could use in-sample residuals, or forecast residuals. A forecaster can be applied to the squared or absolute residuals, so it would be a ResidualUncertaintyForecaster(point_forecaster, uncertainty_forecaster)..
• taking quantile forecaster(s) and composing a distribution based on forecast quantiles. QuantileForecasterComposer or similar.
• naive quantile forecasts: take rolling windows of length n, from these get a time series of quantiles. Apply a forecaster to said quantiles, then apply the quantile composer.
• adding average probability distributions to boosting and weighted averages to ensembling
• boosting probability distributions

[fkiraly]

and, of course, adding proper probability outputs in all the forecasters that already should return intervals, variances, etc.

[eenticott-shell]

Started squared/absolute residual wrapper in #2698.

[sanjayk0508]

taking quantile forecaster(s) and composing a distribution based on forecast quantiles. QuantileForecasterComposer or similar.

@fkiraly, adding on to your suggestions:

Adding parallelization to QuantileForecasterComposer: Given that each quantile forecaster could potentially be trained independently, we can also introduce parallelization to expedite the training process. This could be particularly useful if users will be using a large number of quantile forecasters in the QuantileForecasterComposer. For parallelization, we can use sktime/parallel.py, particularly, _parallelize_joblib and _parallelize_dask

naive quantile forecasts: take rolling windows of length n, from these get a time series of quantiles. Apply a forecaster to said quantiles, then apply the quantile composer.

Adaptive window length: The ability to adjust the rolling window length n based on the data's characteristics can help capture recent changes in the data more accurately, especially in highly varying time series data.

[fkiraly]

Adding parallelization to QuantileForecasterComposer: Given that each quantile forecaster could potentially be trained independently, we can also introduce parallelization to expedite the training process.

Good point! For tabular regressors, @Ram0nB has implemented a MultipleQuantileRegressor in skpro and is indeed using joblib Parallelize:
https://github.com/sktime/skpro/blob/main/skpro/regression/multiquantile.py
(although users can pass only the n_jobs parameter)

Perhaps that estimator might be a good starting point?

One issue with the idea is, also, that there are no "pure quantile forecasters" in sktime, but there are plenty tabular quantile regressors, which can be used via reduction.

[fkiraly]

Adaptive window length: The ability to adjust the rolling window length n based on the data's characteristics can help capture recent changes in the data more accurately, especially in highly varying time series data.

How would that work? Would it be any different from tuning via grid search etc?

[sanjayk0508]

How would that work? Would it be any different from tuning via grid search etc?

I was thinking the adaptive window length feature would monitor the behavior of the data. When the data exhibits significant variability, this would automatically adjust the size of the rolling window accordingly. Which would ensure that recent changes in the data are captured more accurately.

But yes, you are correct this wouldn't actually be any different from tuning via grid search.