Skip to content

rolling_origin

Jump to bottom
Ivan Svetunkov edited this page Jun 24, 2026 · 5 revisions

Rolling Origin — Cross-Validation for Time Series

Function signatures

R

ro(data, h=10, origins=10, call, value=NULL, step=1,
   ci=FALSE, co=TRUE, silent=TRUE, parallel=FALSE, ...)

Python

def rolling_origin(
    data: np.ndarray,
    h: int = 10,
    origins: int = 10,
    step: int = 1,
    ci: bool = False,
    co: bool = True,
    call: Callable | None = None,
    silent: bool = True,
) -> RollingOriginResult: ...

Overview

Rolling origin evaluation (also known as time series cross-validation) produces forecasts from multiple time origins, each time expanding (or sliding) the training window and forecasting h steps ahead. This is the standard approach for evaluating forecast accuracy on time series data where random train/test splits are not appropriate.

Import

# R — function is in the greybox namespace
library(greybox)
# Python
from greybox import rolling_origin, RollingOriginResult

Algorithm

Given a series of length n, the initial in-sample window size is:

obs_in_sample = n - (origins * step + (h - step) * co)

where co is 1 (constant holdout) or 0 (decreasing holdout). This matches R's ro() formula exactly.

Per origin i (0-based):

  • Expanding window (ci=False, default): training indices [0, obs_in_sample + step*i)
  • Sliding window (ci=True): training indices [step*i, obs_in_sample + step*i) (constant size)
  • Holdout: indices [train_end, train_end + h_actual)
    • co=True: h_actual = h (all origins forecast exactly h steps)
    • co=False: h_actual = min(h, step * (origins - i)) (later origins may forecast fewer steps)

Parameters

Parameter R Python Type Default Description
data data data vector / ts / np.ndarray — Time series data
h h h integer / int 10 Forecast horizon
origins origins origins integer / int 10 Number of rolling origins
call call call character / Callable — R: string model call; Python: callable(data, h) → forecasts
step step step integer / int 1 Step between origins
ci ci ci logical / bool FALSE / False Constant in-sample (sliding window)
co co co logical / bool TRUE / True Constant out-of-sample (holdout)
silent — silent — / bool True Suppress progress output

call parameter (Python)

The call parameter is a single callable with signature:

call(data: np.ndarray, h: int, **optional) -> forecasts

data is the training slice, h is the forecast horizon. The function may optionally accept keyword arguments counti, counto, and/or countf (index arrays into the original series) — they are injected automatically if present in the signature:

  • counti — indices of training observations in the original series
  • counto — indices of holdout observations
  • countf — concatenation of counti and counto

Return value (any of these):

Return type Stored as
np.ndarray or list result.mean
dict with string keys one attribute per key
object with .mean / .lower / .upper same as dict
3-tuple (mean, lower, upper) result.mean, result.lower, result.upper

Return Value

RollingOriginResult with shape (h, origins) — rows = forecast step, columns = origin.

Attribute Type Description
.actuals np.ndarray Original data
.holdout (h, origins) float64, NaN-filled Actual holdout values
.mean (h, origins) Point forecasts (always present)
.lower (h, origins) or absent Lower interval bound
.upper (h, origins) or absent Upper interval bound
.origins int Number of origins evaluated
.h int Forecast horizon
._fields list[str] Names of all forecast fields

NaN is used in .holdout and forecast matrices for steps beyond h_actual (co=False near end of series).

__str__ output

Rolling origin with constant holdout was done.
 Forecast horizon: 10
 Number of origins: 10

Examples

Basic Usage

# R
library(greybox)
y <- rnorm(100)
ourCall <- "predict(alm(y~1, data=data), h=5)"
result <- ro(y, h=5, origins=10, call=ourCall)
print(result)
# Python
import numpy as np
from greybox import rolling_origin

np.random.seed(42)
y = np.cumsum(np.random.randn(100))

result = rolling_origin(y, h=5, origins=10,
    call=lambda data, h: np.full(h, data.mean()))

print(result)
# Rolling origin with constant holdout was done.
#  Forecast horizon: 5
#  Number of origins: 10

print("holdout shape:", result.holdout.shape)  # (5, 10)
print("mean shape:",    result.mean.shape)      # (5, 10)

With ALM (greybox model)

import numpy as np
from greybox import rolling_origin, ALM

np.random.seed(42)
y = np.cumsum(np.random.randn(100))

def call_alm(data, h):
    X = np.column_stack([np.ones(len(data)),
                         np.arange(len(data))])
    model = ALM(distribution="dnorm").fit(X, data)
    n = len(data)
    X_new = np.column_stack([np.ones(h), np.arange(n, n + h)])
    return model.predict(X_new, interval="none").mean

result = rolling_origin(y, h=5, origins=10, call=call_alm)
print(result.mean)  # (5, 10) point forecasts

With Prediction Intervals

import numpy as np
from greybox import rolling_origin

y = np.cumsum(np.random.randn(100))

def call_with_intervals(data, h):
    m = np.full(h, data.mean())
    s = data.std()
    return {"mean": m, "lower": m - 1.96 * s, "upper": m + 1.96 * s}

result = rolling_origin(y, h=5, origins=10, call=call_with_intervals)
print("lower:", result.lower.shape)  # (5, 10)
print("upper:", result.upper.shape)  # (5, 10)

With statsmodels ARIMA

import numpy as np
from statsmodels.tsa.arima.model import ARIMA
from greybox import rolling_origin

y = np.cumsum(np.random.randn(200))

def call_arima(data, h):
    model = ARIMA(data, order=(1, 1, 0)).fit()
    fc = model.forecast(h)
    return fc

result = rolling_origin(y, h=10, origins=20, call=call_arima)

Exogenous Regressors via counti / counto

import numpy as np
from greybox import rolling_origin

y = np.cumsum(np.random.randn(200))
x1 = np.random.randn(200)

# counti/counto are injected automatically when in the signature
result = rolling_origin(y, h=10, origins=20,
    call=lambda data, h, counti, counto: (
        np.full(h, np.linalg.lstsq(
            np.column_stack([np.ones_like(data), x1[counti]]),
            data, rcond=None)[0] @ np.column_stack([np.ones(h), x1[counto]]).T
        )
    ))

Expanding vs Sliding Window

# R — expanding window (default)
result_exp <- ro(y, h=5, origins=10, co=FALSE, call=ourCall)

# R — sliding window (constant training size)
result_slide <- ro(y, h=5, origins=10, ci=TRUE, co=FALSE, call=ourCall)
# Python — expanding window (default)
result_exp = rolling_origin(y, h=5, origins=10, ci=False, call=call_alm)

# Python — sliding window (constant training size)
result_slide = rolling_origin(y, h=5, origins=10, ci=True, call=call_alm)

Evaluating Forecasts

import numpy as np
from greybox import rolling_origin

y = np.cumsum(np.random.randn(100))
result = rolling_origin(y, h=5, origins=10,
    call=lambda data, h: np.full(h, data.mean()))

from greybox.point_measures import rmse

# RMSE per forecast step (averaged over origins)
for step in range(result.holdout.shape[0]):
    err = rmse(result.holdout[step], result.mean[step])
    print(f"h={step + 1}: RMSE={err:.4f}")

References

  • Tashman, L.J. (2000). Out-of-sample tests of forecasting accuracy: an analysis and review. International Journal of Forecasting, 16(4), pp.437-450.
  • Svetunkov, I. (2023). Forecasting and Analytics with the Augmented Dynamic Adaptive Model. Chapter 2. https://openforecast.org/adam/

Clone this wiki locally