[ENH] by-horizon forecaster, for different estimator/parameter per horizon

[fkiraly]

Fixes #3018

Implements a forecaster FhPlexForecaster that allows to specify different parameters per forecasting horizon.

To specify different forecasters, combine with MultiplexForecaster or MultiplexTransformer.

Potentially useful in #4776 (comment).

FYI @davidgilbertson, would appreciate feedback.

Currently does not have as features (but should have?)

• multiple fh indices per forecaster
• inheritance from _HeterogenousMetaEstimator - I already prepared the _forecasters attr for that but didn't yet test it with inheritance. Hopefully works out of the box.

[davidgilbertson]

I don't have any strong opinions on this. It does look like it could solve the problem I have, but requires quite a lot of user code.

I think/hope I might be confused because I thought what I'm doing was pretty much stock-standard time series analysis and would be how the default, simple, inflexible would work. Can you confirm whether or not this particular use case should be currently possible?

• I have X and y time series data, with multiple instances.
• I want to put, say, 2 lags of y data into my X dataframe (two new columns) and pass that to a particular estimator's .fit() method.
• I want to predict, say, 14 timesteps in the future

Everything I've tried fails in some way:

• make_reduction by default lags every column in X - a flag to control whether the user wants to lag X, y, both, or some named columns would solve this.
• make_reduction passing transformers and window_length=None can only do 'global' calculations, not one-estimator-per-instance.
• YfromX leaks future values of y into the dataframe passed to my estimator during .predict()
• DirectReductionForecaster takes far too long to run (more than 30x slower)

So is this use case (X and y data, want to put lags of y into X) actually impossible at the moment, and this PR resolves that? Or is there something I've missed.

[fkiraly]

actually impossible at the moment, and this PR resolves that? Or is there something I've missed.

Yes - the idea is to make it possible first (the combination of unlagged X, lagged y, global, which for an odd reason seems not supported) and then optimize.

Tbh the off-shelf reducers should work here.

Btw, it would be great if you could paste some code for DirectReductionForecaster in this issue that shows a long runtime: #3224 - I have been mostly diagnosing the RecursiveReductionForecaster.

[davidgilbertson]

OK I'm glad you agree its odd that it's not supported, that means I'm not going crazy :)

Actually it seems I've just discovered a way. I think I can wrap my whole pipeline in ForecastByLevel, and then use the reduction method that only works with global pooling:

make_reduction(
    HistGradientBoostingRegressor(),
    transformers=[
        WindowSummarizer(
            lag_feature={"lag": range(1, target_lags + 1)},
            truncate="bfill",
            n_jobs=n_jobs,
        ),
    ],
    pooling="global",
    window_length=None,
)

Haven't tested thoroughly, and it still has a tiny leak due to bfill reaching into the future, but it seems that it at least allows me to use sktime.

[fkiraly]

OK I'm glad you agree its odd that it's not supported, that means I'm not going crazy :)

No, not at all.

It is not uncommon that features are added consecutively on estimators (e.g., global, use of X) so that combination cases don't work properly.

That's why we wanted to start with a re-design in 2022, which has the "right user interface" imo, it ended up working, but being computationally too expensive. So the next step is keeping the interface but making it faster.