How to use WindowSummarizer in a TransformedTargetForecaster pipeline

[jpss95]

Hi!, I'm using recursive forecasting using XGBoost and WindowSummarizer. The number of series that i want to predict is large, and I want to make a custom model for each one, so i think about generating a lot of lags and window and then include a FeatureSelection method in my pipeline to select the best lags and windows for each series, but I got some problems while using a pipeline.

As I understood in the examples, using a pipeline with a WindowSummarizer as first step should casuse the same effect as a make_reduction method with a WindowSummarizer transformer inside, but at least in my code it doesn't.

While using the make reduction with the transformer inside the method creates an object that predict the desired series. That's perfect

But when I use the pipeline (TransformedTargetForecaster) the generated object is fitted to each of the features created by the WindowSummarizer

Note: all the lags and windows that i'm trying to create are based on the target.

Here is my code to reproduce the problem:

Get data and basic variables/parameters

import pandas as pd
import numpy as np
from sktime.forecasting.compose import (TransformedTargetForecaster,
                                        make_reduction)
from sktime.transformations.series.summarize import WindowSummarizer
from sktime.forecasting.base import ForecastingHorizon

# Create the monthly Period Index
date_rng = pd.period_range(start='2016-01', end='2023-12', freq='M')

# Generate random data for the demand column
demand_data = np.random.randint(low=100, high=1000, size=(len(date_rng)))

# Create the DataFrame
df = pd.DataFrame(data={'monthly_demand': demand_data}, index=date_rng)

fh = ForecastingHorizon(np.arange(1, 9))

kwargs = {
    "lag_feature": {
        "lag": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12],
        "mean": [[1, 3]],
        "std": [[1, 4]]},
    "truncate": 'bfill'
}

Option 1: Make reduction + window summarizer

forecaster = make_reduction(
    XGBRegressor(),
    transformers=[WindowSummarizer(**kwargs, n_jobs=1, target_cols=['monthly_demand'])],
    window_length=None,
    strategy="recursive",
    pooling="global",
)

forecaster.fit(df, fh=fh)

forecaster.predict(fh=fh)

The result was:

Option 2: pipeline + Make reduction

forecaster = make_reduction(
    XGBRegressor(), 
    scitype = 'tabular-regressor',   
    strategy = 'recursive', 
    window_length = 12) 
pipe_y = TransformedTargetForecaster(
        steps=[
            ("summarizer", WindowSummarizer(**kwargs, n_jobs=1, target_cols=['monthly_demand'])),
            ("forecaster", forecaster),
        ]
    )
pipe_y.fit(y=df, fh=fh)
pipe_y.predict(fh=fh)

The result was a prediction for each of the generated columns by the WindowSummarizer:

Anyone knows how to use the pipeline properly ?

[fkiraly]

Sorry for the late reply, this was a long post!

If I understand correctly, you want to find out how you can manually apply the WindowSummarizer in a pipeline, like it happens in make_reduction, just with any other forecaster, right?

The point is, the TransformedTargetForecaster applies it to y, and replaces y with it, but you want it as X, right?

That would be via YtoX, which moves your y to X, then you can apply the WindowSummarizer - and it has to happen in a ForecastingPipeline that works on X.

Example code:

pipe_y = ForecastingPipeline(
        steps=[
            ("ytox", YtoX()),
            ("summarizer", WindowSummarizer(**kwargs, n_jobs=1, target_cols=['monthly_demand'])),
            ("forecaster", forecaster),
        ]
    )
pipe_y.fit(y=df, fh=fh)
pipe_y.predict(fh=fh)

or, shorter, (YtoX() * WindowSummarizer(stuff)) ** forecaster

note, if you have exogenous data and want to use it instead of replace it with the summary features, you also need to use a FeatureUnion, e.g.,

pipe_y = ForecastingPipeline(
        steps=[
            ("id-plus-ytox", Id() + YtoX()),
            ("summarizer", WindowSummarizer(**kwargs, n_jobs=1, target_cols=['monthly_demand'])),
            ("forecaster", forecaster),
        ]
    )
pipe_y.fit(y=df, fh=fh)
pipe_y.predict(fh=fh)

the Id here keeps your original features. The + has a long version which is FeatureUnion.

If you want to apply the WindowSummarizer only to the y-features, you need to pipe the WindowSummarizer with YtoX.

Short version is ( (YtoX() * WindowSummarizer(stuff)) + Id() ) ** forecaster

[jpss95]

Thank you Franz!

[arabe91]

how would we use additional exogenous data provided as X parameter in the fit and predict function together with the WindowSummarizer which only applies to y? i am bit confused here

[fkiraly]

@arabe91, can you kindly explain what you mean, or what precisely you would like to do?

[fenoy]

I think he means how you introduce an exogenous variable, for example df["exogenous"], into this pipeline. I guess you need to add what you suggested to the exogenous variable with an additional FeatureUnion, but I am not sure about that. If someone find a solution to this, it would be helpful also for me.

[sbuse]

I'm stuck on a similar problem and would love to see more examples of how to use pipelines in combination with WindowSummarizer and make_reduction. Can anyone point me somewhere for more examples?

A big question for me is also the examination of forecasting pipelines. Is there a way to print or show the data with which the regressor inside of make_reduction is trained?

[fenoy]

@fkiraly I think your answer is confusing when you say "note, if you have exogenous data and want to use it instead of replace it with the summary features, you also need to use a FeatureUnion, e.g., ...", but then you do not use exogenous data in fit() nor predict(). I have tried to directly add it and it produces an error.

[fenoy]

By looking at the docs, I found that they do this in an example:

y, X = load_longley()
y_train, y_test, X_train, X_test = temporal_train_test_split(y, X)

fh = ForecastingHorizon(X_test.index, is_relative=False)

Z_train = pd.concat([X_train, y_train], axis=1)
Z_test = pd.concat([X_test, y_test], axis=1)

kwargs = {
    "lag_feature": {
        "lag": [1],
        "mean": [[1, 3], [3, 6]],
        "std": [[1, 4]],
    }
}

pipe = ForecastingPipeline(
    steps=[
        ("a", WindowSummarizer(n_jobs=1, target_cols=["POP", "TOTEMP"])),
        ("b", WindowSummarizer(**kwargs, n_jobs=1, target_cols=["GNP"])),
        ("forecaster", NaiveForecaster(strategy="drift")),
    ]
)

pipe_return = pipe.fit(y_train, Z_train)
y_pred2 = pipe_return.predict(fh=fh, X=Z_test)

I guess it is not the most elegant solution when you could use something resembling what @fkiraly suggested, but at least it works.

Remember to use a lag of at least 1, otherwise, since you are using the target as a predictor, it would result in data leakage.

[shabnamnazmids]

I am having the same problem with concatenating X and y features. One issue with this solution is the availability of a "y_test" in production.