rolling_origin

Rolling Origin — Cross-Validation for Time Series

Overview

Rolling origin evaluation (also known as time series cross-validation) produces forecasts from multiple time origins, each time expanding (or sliding) the training window and forecasting h steps ahead. This is the standard approach for evaluating forecast accuracy on time series data where random train/test splits are not appropriate.

Import

# R — function is in the greybox namespace
library(greybox)

# Python
from greybox.rolling import rolling_origin, RollingOriginResult

Algorithm

1. Start with an initial training window of size n - h - (origins - 1) * step
2. Fit the model on the training data using model_fn
3. Produce h-step-ahead forecasts using predict_fn
4. Record forecasts and corresponding actual values
5. Expand the training window by step observations (or slide if ci=True)
6. Repeat from step 2 until all origins are processed

Parameters

Parameter	R	Python	Type	Default	Description
data	data	data	vector / ts / np.ndarray	—	Time series data
h	h	h	integer / int	10 / 1	Forecasting horizon
origins	origins	origins	integer / int	10 / 5	Number of rolling origins
call	call	—	character / —	—	String with the model call (e.g., "alm(y~x, data=data)")
value	value	—	character / —	NULL	Which value to extract from the model
step	step	step	integer / int	1	Step between origins
ci	ci	ci	logical / bool	FALSE / False	Constant in-sample (sliding window)
co	co	co	logical / bool	TRUE / False	Constant out-of-sample (holdout) window
model_fn	—	model_fn	— / Callable	None	Function: train_data -> fitted_model
predict_fn	—	predict_fn	— / Callable	None	Function: (model, h) -> forecasts
silent	—	silent	— / bool	True	Suppress progress output

Return Value

Field	R	Python	Type	Description
holdout	$holdout	.holdout	matrix / np.ndarray	Matrix of holdout values (origins x h)
forecasts	$forecasts	.forecasts	matrix / dict	Matrix of forecasts, or dict keyed by "origin_0", etc.
origins	$origins	.origins	integer / int	Number of origins evaluated
actuals	—	.actuals	— / np.ndarray	The original data
h	—	.h	— / int	Forecasting horizon

Key Differences

Feature	R (ro)	Python (rolling_origin)
Model specification	String-based call	Callback functions model_fn / predict_fn
Default origins	10	5
Default horizon	10	1
Default co	TRUE	False
Forecasts output	Matrix	Dictionary of arrays

Examples

Basic Usage

# R
library(greybox)

y <- rnorm(100)
ourCall <- "predict(alm(y~1, data=data), h=5)"
result <- ro(y, h=5, origins=10, call=ourCall)
print(result)

# Python
import numpy as np
from greybox.alm import ALM
from greybox.rolling import rolling_origin

np.random.seed(42)
y = np.cumsum(np.random.randn(50))

def model_fn(train_data):
    """Fit a simple model on the training data."""
    X = np.column_stack([np.ones(len(train_data)),
                         np.arange(len(train_data))])
    model = ALM(distribution="dnorm")
    return model.fit(X, train_data)

def predict_fn(model, h):
    """Produce h-step-ahead forecasts."""
    n = len(model.actuals)
    X_new = np.column_stack([np.ones(h),
                             np.arange(n, n + h)])
    return model.predict(X_new, interval="none").mean

result = rolling_origin(y, h=5, origins=3,
                        model_fn=model_fn,
                        predict_fn=predict_fn)

print(f"Origins evaluated: {result.origins}")
print(f"Holdout shape: {result.holdout.shape}")

Expanding vs Sliding Window

# R — expanding window (default)
result_exp <- ro(y, h=5, origins=10, co=FALSE, call=ourCall)

# R — sliding window (constant training size)
result_slide <- ro(y, h=5, origins=10, ci=TRUE, co=FALSE, call=ourCall)

# Python — expanding window (default) — training set grows at each origin
result_exp = rolling_origin(y, h=5, origins=10, ci=False,
                            model_fn=model_fn, predict_fn=predict_fn)

# Python — sliding window — training set has constant size
result_slide = rolling_origin(y, h=5, origins=10, ci=True,
                              model_fn=model_fn, predict_fn=predict_fn)

Evaluating Forecasts

# R
for (i in 1:result$origins) {
  mae_val <- mean(abs(result$holdout[i,] - result$forecasts[i,]))
  cat("Origin", i, ": MAE=", mae_val, "\n")
}

# Python
import numpy as np
from greybox.measures import mae, rmse

# Compare forecasts against holdout for each origin
for i in range(result.origins):
    fc = result.forecasts[f"origin_{i}"]
    ho = result.holdout[i]
    print(f"Origin {i}: MAE={mae(ho, fc):.4f}, RMSE={rmse(ho, fc):.4f}")

References

• Tashman, L.J. (2000). Out-of-sample tests of forecasting accuracy: an analysis and review. International Journal of Forecasting, 16(4), pp.437-450.
• Svetunkov, I. (2023). Statistics for Business Analytics. https://openforecast.org/sba/