Add time series forecasting support

[Lopa10ko]

WIP

• refactor code after initial review
• add pip install fedot[extra] to a setup file
• manually test fedot ts forecasting scenario

closes #610

[shchur]

Nit: f-strings are a bit more readable and are compatible with all Python versions used by AMLB

f"Running FEDOT with a maximum time of {config.max_runtime_seconds}s on {n_jobs} cores, optimizing {scoring_metric}"

[Lopa10ko]

d583cb2

[shchur]

This can be very inefficient for large dataframes with many time series (scales as O(N^2)). A better option would be to use groupby, e.g.

for label, ts in train_df.groupby(id_column, sort=False):
    train_series = ts[dataset.target].to_numpy()

[Lopa10ko]

thanks, tried that: imo using zip may be inappropriate here, so I would stick with a janky bad-asymptotic solution to ensure that the labels are matching
e6f19e7

[shchur]

This code uses the range index [0, 1, 2, ..., n-1] instead of the timestamps of the time series. Should this be train_sub_df[timestamp_column] instead, or does FEDOT ignore the timestamps?

[Lopa10ko]

FEDOT ignores timestamps, so that idx should be fine

[shchur]

This contains the future values of the time series, I doubt that these values need to be fed to the model at predictions time.

For example, if training data contains time steps [1, 2, 3, ..., T] and the goal is to predict [T+1, ..., T+H], then based on your current code

• train_series contains timesteps [1, 2, 3, ..., T]
• test_series contains timesteps [T+1, ..., T+H]

My guess is that we need to pass train_series as input to both fit() and predict()

[Lopa10ko]

in essence predict and forecast are the same thing, but we could pass test_input with features from train_series and target from test_series to get the prediction horizon in a predict

changed to a clear forecast method
a357726

[shchur]

Why is / 10 necessary here?

[Lopa10ko]

generally speaking, this is a small safety measure to ensure that the training time of one pipeline is exactly within the total timeout. the classification and regression #563 uses the same empirical approach. it should be patched in the future

[shchur]

Nit: I think it would be cleaner to remove the line training_duration, predict_duration = 0, 0 above and just return training_duration=training.duration, predict_duration=predict.duration here.

[Lopa10ko]

since we train the FEDOT model for each series (for each label), training.duration value isn't cumulative and will only reflect the time spent on the last iteration

[shchur]

My bad, I didn't realize this was happening inside the loop over individual series

[shchur]

It's necessary to return dataset.repeated_item_id and dataset.repeated_abs_seasonal_error as optional_columns in the result for MASE computation to work correctly (see https://github.com/openml/automlbenchmark/blob/master/frameworks/AutoGluon/exec_ts.py#L63C1-L67C1).
This is a rather ugly hack that is necessary to make history-dependent metrics like MASE compatible with the AMLB results API.

[Lopa10ko]

c400961

[Lopa10ko]

@shchu, thanks for the initial review :)
I intend to continue exploring the benchmarks and testing some things locally in order to ensure that all FEDOT-related functionality is working correctly. This may take some time, so when I'm finished, I will undraft this pull request.