Multi Forecasting Features

[protti]

Hello to everyone,
I want to ask if is possible to adopt the solution in SNARIMAX for multiple forecasting features.
In the example of SNARIMAX, the y is only composed by one features:

for x, y in datasets.AirlinePassengers():
    y_pred = model.forecast(horizon=1, xs=[x])
    model = model.learn_one(x,y)
    metric = metric.update(y, y_pred[0])

My goal is to give to the function learn_one(x,y) a multiple features of y.
Something like this:

for x, y in datasets.AirlinePassengers():
    y_pred = model.forecast(horizon=1, xs=[x])
    model = model.learn_one(x,[y[0],y[1]])
    metric = metric.update([y[0],y[1]], [y_pred[0],y_pred[1]])

In order to forecast not one single features but more features.
Hope to be clear and thanks for the answer!

[raphaelsty]

Hi @protti,

I'm not sure I fully understand what you are looking for. If you want to learn from several target variables, the multioutput.RegressorChain module is not compatible with SNARIMAX. But you can create two separate models and store them into a dict if you have two different target variables.

Regarding metrics, the module metrics.RegressionMultiOutput offers what you are looking for. The input parameters are dictionaries and not lists. :)

from river import metrics

metric = metrics.RegressionMultiOutput(metrics.MAE())

metric.update(
    y_pred = {"y_1": y_pred_1, "y_2": y_pred_2}, 
    y_true = {"y_1": y_1, "y_2": y_2}, 
)

[protti]

Thanks for the answer @raphaelsty.
In bref, my idea is to have a Single Incremental Regressor that take in input multiple Features, like this:

# Incremental Input to the Regressor
{'moment': datetime.datetime(2016, 4, 1, 15, 43, 12), 'clouds': 90,  'humidity': 66, 'pressure': 1019.0, 'temperature': 9.7, 'wind': 5.1}
{'moment': datetime.datetime(2016, 4, 1, 15, 44, 15), 'clouds': 90, 'humidity': 70, 'pressure': 1019.0, 'temperature': 9.06, 'wind': 1.0}
{'moment': datetime.datetime(2016, 4, 1, 15, 45, 15), 'clouds': 91, 'humidity': 75, 'pressure': 1021.0, 'temperature': 9.4, 'wind': 2.0}
...

And try to forecast each features used as input:

# Forecast of new Features Values
{'moment': datetime.datetime(2016, 4, 1, 15, 46, 15), 'clouds': 91,  'humidity': 74, 'pressure': 1019.0, 'temperature': 9.2, 'wind': 4.0}

[raphaelsty]

You can pass your features to SNARIMAX but is not designed for multi-output regression. 🙂 In the sample of code below, I slightly updated the tutorial of SNARIMAX to create multiples models. You can assign a single target to each model. It would look like this:

import calendar
import datetime as dt
from river import compose
from river import datasets
from river import linear_model
from river import metrics
from river import optim
from river import preprocessing
from river import time_series
import math
import copy

def get_month_distances(x):
    return {
        calendar.month_name[month]: math.exp(-(x['month'].month - month) ** 2)
        for month in range(1, 13)
    }

def get_ordinal_date(x):
    return {'ordinal_date': x['month'].toordinal()}

extract_features = compose.TransformerUnion(
    get_ordinal_date,
    get_month_distances
)

model = (
    extract_features |
    time_series.SNARIMAX(
        p=0,
        d=0,
        q=0,
        m=12,
        sp=3,
        sq=6,
    regressor=(
         preprocessing.StandardScaler() |
         linear_model.LinearRegression(
             intercept_init=110,
             optimizer=optim.SGD(0.01),
             intercept_lr=0.3
         )
     )
    )
)


models = {}
for name in ['clouds',  'humidity', 'pressure', 'temperature', 'wind']:
    models[name] = copy.deepcopy(model)
    
metric = metrics.RegressionMultiOutput(metrics.Rolling(metrics.MAE(), 12))

for x, y in your_stream:
    
    # x would be your set of features {"moment": datetime, "clouds": xxx, "humidity": xxx, "pressure": xxx, "temperature": xxx, "wind": xxx}
    # Each value of x is set at horizon 0, i.e feature available now.
    # Here y is a dict {"clouds": yyy, "humidity": yyy, "pressure": yyy, "temperature": yyy, "wind": yyy}
    # Each value of y is set at horizon 1, i.e the next observation.
    
    y_preds = {}
    for name, model in models.items():
        y_preds[name] = model.forecast(horizon=1, xs=[x])[0]
        model.learn_one(x, y[name])
        
    metric.update(y, y_preds)

Also, I do not understand why you would forecast the field "moment"

[protti]

Hello @raphaelsty , thanks again for your answer.
Your solution is the same that I've applied at the end. My idea to use one single model for every feature derives from the fact that putting all the features in one model can improve the quality of the forecasting. Therefore, I would like to use one single model that contains all the features.
For momentum, I've just made a mistake.
Sorry for it and thanks for your solution!

[raphaelsty]

@protti I think this is the right way to do it with SNARIMAX, in the example I shared, all the models have the same features. Only the target of the model differs. :)

To go further, you can feed the features of model 2 with the prediction of model 1 and feed the features of model 3 with the predictions of models 1 and 2 etc... This would be like creating a chain regressor.

for x, y in your_stream:
    
    # x would be your set of features {"moment": datetime, "clouds": xxx, "humidity": xxx, "pressure": xxx, "temperature": xxx, "wind": xxx}
    # Each value of x is set at horizon 0, i.e feature available now.
    # Here y is a dict {"clouds": yyy, "humidity": yyy, "pressure": yyy, "temperature": yyy, "wind": yyy}
    # Each value of y is set at horizon 1, i.e the next observation.
    
    y_preds = {}
    for name, model in models.items():
        y_preds[name] =  model.forecast(horizon=1, xs=[x])[0]
        model.learn_one(x, y[name])

        # Adding previous models prediction as a feature of the next model.
        x[f"forecast_{name}"] = y_preds[name] 
        
        
    metric.update(y, y_preds)