fitted values on re-fitted model seem to be affected by new data

[robjhyndman]

library(fpp2)
# Create training and two sets of test data
training <- subset(auscafe, end=length(auscafe)-61)
test2 <- test <- subset(auscafe, start=length(auscafe)-60)
test2[61] <- test[61] + 2

# Apply same model to all three time series
cafe.train <- Arima(training, order=c(2,1,1), seasonal=c(0,1,2), lambda=0)
cafe.test <- Arima(test, model=cafe.train)
cafe.test2 <- Arima(test2, model=cafe.train)

# Fitted values on each test set
cafe.test.fit <- fitted(cafe.test)
cafe.test2.fit <- fitted(cafe.test2)

window(cafe.test.fit, c(2017,1), c(2017,9)) 
#>           Jan      Feb      Mar      Apr      May      Jun      Jul
#> 2017 3.639529 3.313001 3.614441 3.571203 3.588871 3.483909 3.760809
#>           Aug      Sep
#> 2017 3.780694 3.748437
window(cafe.test2.fit, c(2017,1), c(2017,9))
#>           Jan      Feb      Mar      Apr      May      Jun      Jul
#> 2017 3.639529 3.313001 3.614441 3.571203 3.588871 3.483909 3.760809
#>           Aug      Sep
#> 2017 3.780694 3.812552

^{Created on 2019-05-01 by the reprex package (v0.2.1)}

Why does the last fitted value change?

[Steviey]

+1

[mitchelloharawild]

I get the same result in fable - why is the change in fitted values unexpected?

[robjhyndman]

Because a fitted value is a one-step forecast, and should not be affected by the following observation.

[mitchelloharawild]

MRE directly from stats::arima(). Seems to only occur when a SMA is included in the model.

It also seems to happen within ARIMA_Like, which updates the model based on data when computing the fit - will need to dig deeper into this later.

# Create training and two sets of test data
training <- USAccDeaths[1:60]
test2 <- test <- USAccDeaths[61:72]
test2[12] <- test[12] + 2

fit <- stats::arima(log(training), order = c(0,1,0), seasonal = c(0,1,1))
refit1 <- stats::arima(log(test), order = c(0,1,0), seasonal = c(0,1,1), fixed = coef(fit))
refit2 <- stats::arima(log(test2), order = c(0,1,0), seasonal = c(0,1,1), fixed = coef(fit))

exp((log(test) - refit1$residuals)[10:12])
#> [1] 9270.875 9187.616 8740.719
exp((log(test2) - refit2$residuals)[10:12])
#> [1] 9270.875 9187.616 8740.807

^{Created on 2019-06-04 by the reprex package (v0.2.1)}