rolling_origin

Rolling Origin — Cross-Validation for Time Series

Overview

Rolling origin evaluation (also known as time series cross-validation) produces forecasts from multiple time origins, each time expanding (or sliding) the training window and forecasting h steps ahead. This is the standard approach for evaluating forecast accuracy on time series data where random train/test splits are not appropriate.

Import

# R — function is in the greybox namespace
library(greybox)

# Python
from greybox.rolling import rolling_origin, RollingOriginResult

Algorithm

1. Start with an initial training window of size n - h - (origins - 1) * step
2. Fit the model on the training data using model_fn
3. Produce h-step-ahead forecasts using predict_fn
4. Record forecasts and corresponding actual values
5. Expand the training window by step observations (or slide if ci=True)
6. Repeat from step 2 until all origins are processed

Parameters

R: ro()

Parameter	Type	Default	Description
data	vector / ts	—	Time series data
h	integer	10	Forecasting horizon
origins	integer	10	Number of rolling origins
call	character	—	String with the model call (e.g., "alm(y~x, data=data)")
value	character	NULL	Which value to extract from the model (e.g., "forecast")
step	integer	1	Step between origins
ci	logical	FALSE	Constant in-sample (sliding window)
co	logical	TRUE	Constant out-of-sample (holdout) window

Python: rolling_origin()

Parameter	Type	Default	Description
data	np.ndarray	—	Time series data
h	int	1	Forecasting horizon
origins	int	5	Number of rolling origins
step	int	1	Step between origins
ci	bool	False	Constant in-sample (sliding window). False = expanding window
co	bool	False	Constant out-of-sample window
model_fn	Callable	None	Function: train_data -> fitted_model
predict_fn	Callable	None	Function: (model, h) -> forecasts
silent	bool	True	Suppress progress output

Return Value

R: rollingOrigin object

Field	Description
$holdout	Matrix of holdout values
$forecasts	Matrix of forecasts
$origins	Number of origins evaluated

Python: RollingOriginResult

Attribute	Type	Description
.actuals	np.ndarray	The original data
.holdout	np.ndarray	Matrix of actual holdout values (origins x h)
.forecasts	dict	Dictionary of forecast arrays keyed by "origin_0", "origin_1", etc.
.origins	int	Number of origins evaluated
.h	int	Forecasting horizon

Key Differences

Feature	R (ro)	Python (rolling_origin)
Model specification	String-based call	Callback functions model_fn / predict_fn
Default origins	10	5
Default horizon	10	1
Default co	TRUE	False
Forecasts output	Matrix	Dictionary of arrays

Examples

Basic Usage

# R
library(greybox)

y <- rnorm(100)
ourCall <- "predict(alm(y~1, data=data), h=5)"
result <- ro(y, h=5, origins=10, call=ourCall)
print(result)

# Python
import numpy as np
from greybox.alm import ALM
from greybox.rolling import rolling_origin

np.random.seed(42)
y = np.cumsum(np.random.randn(50))

def model_fn(train_data):
    """Fit a simple model on the training data."""
    X = np.column_stack([np.ones(len(train_data)),
                         np.arange(len(train_data))])
    model = ALM(distribution="dnorm")
    return model.fit(X, train_data)

def predict_fn(model, h):
    """Produce h-step-ahead forecasts."""
    n = len(model.actuals)
    X_new = np.column_stack([np.ones(h),
                             np.arange(n, n + h)])
    return model.predict(X_new, interval="none").mean

result = rolling_origin(y, h=5, origins=3,
                        model_fn=model_fn,
                        predict_fn=predict_fn)

print(f"Origins evaluated: {result.origins}")
print(f"Holdout shape: {result.holdout.shape}")

Expanding vs Sliding Window

# Expanding window (default) — training set grows at each origin
result_exp = rolling_origin(y, h=5, origins=10, ci=False,
                            model_fn=model_fn, predict_fn=predict_fn)

# Sliding window — training set has constant size
result_slide = rolling_origin(y, h=5, origins=10, ci=True,
                              model_fn=model_fn, predict_fn=predict_fn)

Evaluating Forecasts

import numpy as np
from greybox.measures import mae, rmse

# Compare forecasts against holdout for each origin
for i in range(result.origins):
    fc = result.forecasts[f"origin_{i}"]
    ho = result.holdout[i]
    print(f"Origin {i}: MAE={mae(ho, fc):.4f}, RMSE={rmse(ho, fc):.4f}")

References

• Tashman, L.J. (2000). Out-of-sample tests of forecasting accuracy: an analysis and review. International Journal of Forecasting, 16(4), pp.437-450.
• Svetunkov, I. (2023). Statistics for Business Analytics. https://openforecast.org/sba/