Time series cross-validation and VAR: Principles of forecasting

[nickcox896]

I am trying to apply time series cross validation for, say h=8 , using two VAR models, in order to get
the measures of forecast accuracy for the test data.

I have managed to set up the following code

library(fpp3)
df_tr <- us_change |>
  stretch_tsibble(.init =100, .step = 1) |>
  relocate(Quarter, Consumption, Income, .id)

df_tr  |>
  model(m1 = VAR(vars(Consumption, Income)~AR(2)),
m2 = VAR(vars(Consumption, Income)~AR(3))) |>
  forecast(h = 8) |>
  group_by(.id,.model) |>
  mutate(h = row_number()) |>
  ungroup() |>
  as_fable(response = ".mean[,1]", distribution = .mean[,1])

But I think that I have not set up correctly the elements of as_fable()

[robjhyndman]

library(fpp3)
#> ── Attaching packages ──────────────────────────────────────────── fpp3 1.0.3 ──
#> ✔ tibble      3.3.1     ✔ tsibble     1.2.0
#> ✔ dplyr       1.2.0     ✔ tsibbledata 0.4.1
#> ✔ tidyr       1.3.2     ✔ ggtime      0.2.0
#> ✔ lubridate   1.9.5     ✔ feasts      0.5.0
#> ✔ ggplot2     4.0.2     ✔ fable       0.5.0
#> ── Conflicts ───────────────────────────────────────────────── fpp3_conflicts ──
#> ✖ lubridate::date()    masks base::date()
#> ✖ dplyr::filter()      masks stats::filter()
#> ✖ tsibble::intersect() masks base::intersect()
#> ✖ tsibble::interval()  masks lubridate::interval()
#> ✖ dplyr::lag()         masks stats::lag()
#> ✖ tsibble::setdiff()   masks base::setdiff()
#> ✖ tsibble::union()     masks base::union()

df_tr <- us_change |>
  stretch_tsibble(.init =100, .step = 1) |>
  relocate(Quarter, Consumption, Income, .id)

fc <- df_tr  |>
  model(m1 = VAR(vars(Consumption, Income)~AR(2)),
        m2 = VAR(vars(Consumption, Income)~AR(3))) |>
  forecast(h = 8) |>
  group_by(.id,.model) |>
  mutate(h = row_number()) |>
  ungroup() |>
  as_fable(response = c("Consumption", "Income"), distribution = .distribution)

accuracy(fc, us_change, by_model = TRUE) 
#> Warning: The future dataset is incomplete, incomplete out-of-sample data will be treated as missing. 
#> 8 observations are missing between 2019 Q3 and 2021 Q2
#> # A tibble: 4 × 11
#>   .model .response   .type      ME  RMSE   MAE   MPE  MAPE  MASE RMSSE   ACF1
#>   <chr>  <chr>       <chr>   <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>  <dbl>
#> 1 m1     Consumption Test  -0.106  0.473 0.363 -27.4  89.9 0.585 0.564  0.456
#> 2 m1     Income      Test  -0.0541 0.897 0.590  45.6 141.  0.628 0.690 -0.250
#> 3 m2     Consumption Test  -0.0829 0.453 0.350 -24.1  83.5 0.565 0.540  0.435
#> 4 m2     Income      Test  -0.0290 0.893 0.593  43.6 136.  0.632 0.687 -0.265

^{Created on 2026-04-03 with reprex v2.1.1}

[nickcox896]

Thank you !