09-nonstationarity.Rmd

---
title: "Non-Stationary Time Series"
subtitle: "EC 421, Set 9"
author: "Edward Rubin"
date: "`r format(Sys.time(), '%d %B %Y')`"
output:
  xaringan::moon_reader:
    css: ['default', 'metropolis', 'metropolis-fonts', 'my-css.css']
    # self_contained: true
    nature:
      highlightStyle: github
      highlightLines: true
      countIncrementalSlides: false
---
class: inverse, middle

```{R, setup, include = F}
options(htmltools.dir.version = FALSE)
library(pacman)
p_load(
  broom, here, tidyverse,
  latex2exp, ggplot2, ggthemes, viridis, extrafont, gridExtra,
  kableExtra,
  data.table,
  dplyr,
  lubridate,
  magrittr, knitr, parallel
)
# Define pink color
red_pink <- "#e64173"
turquoise <- "#20B2AA"
grey_light <- "grey70"
grey_mid <- "grey50"
grey_dark <- "grey20"
# Dark slate grey: #314f4f
# Knitr options
opts_chunk$set(
  comment = "#>",
  fig.align = "center",
  fig.height = 7,
  fig.width = 10.5,
  warning = F,
  message = F
)
opts_chunk$set(dev = "svg")
options(device = function(file, width, height) {
  svg(tempfile(), width = width, height = height)
})
# A blank theme for ggplot
theme_empty <- theme_bw() + theme(
  line = element_blank(),
  rect = element_blank(),
  strip.text = element_blank(),
  axis.text = element_blank(),
  plot.title = element_blank(),
  axis.title = element_blank(),
  plot.margin = structure(c(0, 0, -0.5, -1), unit = "lines", valid.unit = 3L, class = "unit"),
  legend.position = "none"
)
theme_simple <- theme_bw() + theme(
  line = element_blank(),
  panel.grid = element_blank(),
  rect = element_blank(),
  strip.text = element_blank(),
  axis.text.x = element_text(size = 18, family = "STIXGeneral"),
  axis.text.y = element_blank(),
  axis.ticks = element_blank(),
  plot.title = element_blank(),
  axis.title = element_blank(),
  # plot.margin = structure(c(0, 0, -1, -1), unit = "lines", valid.unit = 3L, class = "unit"),
  legend.position = "none"
)
theme_axes_math <- theme_void() + theme(
  text = element_text(family = "MathJax_Math"),
  axis.title = element_text(size = 22),
  axis.title.x = element_text(hjust = .95, margin = margin(0.15, 0, 0, 0, unit = "lines")),
  axis.title.y = element_text(vjust = .95, margin = margin(0, 0.15, 0, 0, unit = "lines")),
  axis.line = element_line(
    color = "grey70",
    size = 0.25,
    arrow = arrow(angle = 30, length = unit(0.15, "inches")
  )),
  plot.margin = structure(c(1, 0, 1, 0), unit = "lines", valid.unit = 3L, class = "unit"),
  legend.position = "none"
)
theme_axes_serif <- theme_void() + theme(
  text = element_text(family = "MathJax_Main"),
  axis.title = element_text(size = 22),
  axis.title.x = element_text(hjust = .95, margin = margin(0.15, 0, 0, 0, unit = "lines")),
  axis.title.y = element_text(vjust = .95, margin = margin(0, 0.15, 0, 0, unit = "lines")),
  axis.line = element_line(
    color = "grey70",
    size = 0.25,
    arrow = arrow(angle = 30, length = unit(0.15, "inches")
  )),
  plot.margin = structure(c(1, 0, 1, 0), unit = "lines", valid.unit = 3L, class = "unit"),
  legend.position = "none"
)
theme_axes <- theme_void() + theme(
  text = element_text(family = "Fira Sans Book"),
  axis.title = element_text(size = 18),
  axis.title.x = element_text(hjust = .95, margin = margin(0.15, 0, 0, 0, unit = "lines")),
  axis.title.y = element_text(vjust = .95, margin = margin(0, 0.15, 0, 0, unit = "lines")),
  axis.line = element_line(
    color = grey_light,
    size = 0.25,
    arrow = arrow(angle = 30, length = unit(0.15, "inches")
  )),
  plot.margin = structure(c(1, 0, 1, 0), unit = "lines", valid.unit = 3L, class = "unit"),
  legend.position = "none"
)
theme_set(theme_gray(base_size = 20))
```

# Prologue

---
name: schedule

# Schedule

## Last Time

Autocorrelation

## Today

- Brief introduction to nonstationarity

## Upcoming

- **Assignment** due Saturday.
- **Assignment** soon after.
---
layout: false
class: inverse, middle
# Nonstationarity
---
layout: true
name: intro
# Nonstationarity
---

## Intro

Let's go back to our assumption of .hi[weak dependence/persistence]

> 1. **Weakly persistent outcomes**—essentially, $x_{t+k}$ in the distant period $t+k$ weakly correlates with $x_t$ (when $k$ is "big").

--

We're essentially saying we need the time series $x$ to behave.

--

We'll define this *good behavior* as .hi[stationarity].
---

## Stationarity

Requirements for .hi[stationarity] (a *stationary* time-series process):

--

1. The .hi[mean] of the distribution is independent of time, _i.e._,
.center[
$\mathop{\boldsymbol{E}}\left[ x_t \right]  = \mathop{\boldsymbol{E}}\left[ x_{t-k} \right]$ for all $k$
]
--

2. The .hi[variance] of the distribution is independent of time, _i.e._,
.center[
$\mathop{\text{Var}} \left( x_t \right) = \mathop{\text{Var}} \left( x_{t-k} \right)$ for all $k$
]

--
3. The .hi[covariance] between $x_t$ and $x_{t-k}$ depends only on $k$—.pink[not on] $\color{#e64173}{t}$, _i.e._,
.center[
$\mathop{\text{Cov}} \left( x_t,\,x_{t-k} \right) = \mathop{\text{Cov}} \left( x_s,\, x_{s-k} \right)$ for all $t$ and $s$
]

---
name: walks

## Random walks

.hi[Random walks] are a famous example of a nonstationary process:

--

$$
\begin{align}
  x_t = x_{t-1} + \varepsilon_t
\end{align}
$$

--

Why?
--
 $\mathop{\text{Var}} \left( x_t \right) = t \sigma_\varepsilon^2$, which .pink[violates stationary variance].

--

$$
\begin{align}
   \mathop{\text{Var}} \left( x_t \right)
   &= \mathop{\text{Var}} \left( x_{t-1} + \varepsilon_t \right) \\
   &= \mathop{\text{Var}} \left( x_{t-2} + \varepsilon_{t-1} + \varepsilon_t \right) \\
   &= \mathop{\text{Var}} \left( x_{t-3} + \varepsilon_{t-2} + \varepsilon_{t-1} + \varepsilon_t \right) \\
   &\cdots \\
   &= \mathop{\text{Var}} \left( x_0 + \varepsilon_1 + \cdots + \varepsilon_{t_2} + \varepsilon_{t-1} + \varepsilon_t \right) \\
   &= \sigma^2_\varepsilon + \cdots + \sigma^2_\varepsilon + \sigma^2_\varepsilon + \sigma^2_\varepsilon \\
   &= t \sigma^2_\varepsilon
\end{align}
$$


---
layout: false
class: clear, middle

**Q:** What's the big deal with this violation?
---
class: clear

.hi-slate[One 100-period random walk]
```{R, walk1, echo = F}
set.seed(1246)
walk1 <- tibble(x = cumsum(rnorm(1e2)), t = 1:1e2, walk = "1")
walk2 <- tibble(x = cumsum(rnorm(1e2)), t = 1:1e2, walk = "2")
walk3 <- tibble(x = cumsum(rnorm(1e2)), t = 1:1e2, walk = "3")
walk4 <- tibble(x = cumsum(rnorm(1e2)), t = 1:1e2, walk = "4")
walk5 <- tibble(x = cumsum(rnorm(1e2)), t = 1:1e2, walk = "5")
ggplot(data = walk1, aes(x = t, y = x)) +
  geom_hline(yintercept = 0, color = "grey85", size = 1.25) +
  geom_path() +
  theme_axes_math
```
---
class: clear

.hi-slate[Two 100-period random walks]
```{R, walk2, echo = F}
ggplot(data = bind_rows(walk1, walk2), aes(x = t, y = x, color = walk)) +
  geom_hline(yintercept = 0, color = "grey85", size = 1.25) +
  geom_path() +
  scale_color_viridis(option = "magma", discrete = T, begin = 0.15, end = 0.85) +
  theme_axes_math
```
---
class: clear

.hi-slate[Three 100-period random walks]
```{R, walk3, echo = F}
ggplot(data = bind_rows(walk1, walk2, walk3), aes(x = t, y = x, color = walk)) +
  geom_hline(yintercept = 0, color = "grey85", size = 1.25) +
  geom_path() +
  scale_color_viridis(option = "magma", discrete = T, begin = 0.15, end = 0.85) +
  theme_axes_math
```
---
class: clear

.hi-slate[Four 100-period random walks]
```{R, walk4, echo = F}
ggplot(data = bind_rows(walk1, walk2, walk3, walk4), aes(x = t, y = x, color = walk)) +
  geom_hline(yintercept = 0, color = "grey85", size = 1.25) +
  geom_path() +
  scale_color_viridis(option = "magma", discrete = T, begin = 0.15, end = 0.85) +
  theme_axes_math
```
---
class: clear

.hi-slate[Five 100-period random walks]
```{R, walk5, echo = F}
ggplot(data = bind_rows(walk1, walk2, walk3, walk4, walk5), aes(x = t, y = x, color = walk)) +
  geom_hline(yintercept = 0, color = "grey85", size = 1.25) +
  geom_path() +
  scale_color_viridis(option = "magma", discrete = T, begin = 0.15, end = 0.85) +
  theme_axes_math
```
---
class: clear

.hi-slate[Fifty 100-period random walks]
```{R, walk50, echo = F, cache = T}
# Set seed
set.seed(1246)
# Generate data
walk_df <- lapply(X = 1:50, FUN = function(i) {
  tibble(x = cumsum(rnorm(1e2)), t = 1:1e2, walk = as.character(i))
}) %>% bind_rows()
# Plot
ggplot(data = walk_df, aes(x = t, y = x, color = walk)) +
  geom_hline(yintercept = 0, color = "grey85", size = 1.25) +
  geom_path(size = 0.25) +
  scale_color_viridis(option = "magma", discrete = T, end = 0.95) +
  theme_axes_math
```
---
class: clear

.hi-slate[1,000 100-period random walks]
```{R, walk1000, echo = F, cache = T}
# Set seed
set.seed(1246)
# Generate data
walk_df <- lapply(X = 1:1e3, FUN = function(i) {
  tibble(x = cumsum(rnorm(1e2)), t = 1:1e2, walk = as.character(i))
}) %>% bind_rows()
# Plot
ggplot(data = walk_df, aes(x = t, y = x, color = walk)) +
  geom_hline(yintercept = 0, color = "grey85", size = 1.25) +
  geom_path(size = 0.25) +
  scale_color_viridis(option = "magma", discrete = T, end = 0.95) +
  theme_axes_math
```
---
# Nonstationarity
## Problem

*One* problem is that nonstationary processes can lead to .hi[spurious] results.

--

>**Defintion:** .hi[Spurious]
>- not being what it purports to be; false or fake
>- apparently but not actually valid

--

Back in 1974, Granger and Newbold showed that when they **generated random walks** and **regressed the random walks on each other**, .hi[77/100 regressions were statistically significant] at the 5% level (should have been approximately 5/100).

---
class: clear

.hi-slate[Granger and Newbold simulation example:] _t_ statistic ≈ `r lm(filter(walk_df, walk == 1)$x ~ filter(walk_df, walk == 2)$x) %>% summary %>% coef %>% extract(2,3) %>% round(2)`

```{R, gb12, echo = F}
ggplot(data = walk_df %>% filter(walk %in% c(1:2)), aes(x = t, y = x, color = walk)) +
  geom_hline(yintercept = 0, color = "grey85", size = 1.25) +
  geom_path() +
  scale_color_viridis(option = "magma", discrete = T, begin = 0.15, end = 0.85) +
  theme_axes_math
```
---
class: clear

.hi-slate[Granger and Newbold simulation example:] _t_ statistic ≈ `r lm(filter(walk_df, walk == 3)$x ~ filter(walk_df, walk == 4)$x) %>% summary %>% coef %>% extract(2,3) %>% round(2)`

```{R, gb34, echo = F}
ggplot(data = walk_df %>% filter(walk %in% c(3:4)), aes(x = t, y = x, color = walk)) +
  geom_hline(yintercept = 0, color = "grey85", size = 1.25) +
  geom_path() +
  scale_color_viridis(option = "magma", discrete = T, begin = 0.15, end = 0.85) +
  theme_axes_math
```
---
class: clear

.hi-slate[Granger and Newbold simulation example:] _t_ statistic ≈ `r lm(filter(walk_df, walk == 5)$x ~ filter(walk_df, walk == 6)$x) %>% summary %>% coef %>% extract(2,3) %>% round(2)`

```{R, gb56, echo = F}
ggplot(data = walk_df %>% filter(walk %in% c(5:6)), aes(x = t, y = x, color = walk)) +
  geom_hline(yintercept = 0, color = "grey85", size = 1.25) +
  geom_path() +
  scale_color_viridis(option = "magma", discrete = T, begin = 0.15, end = 0.85) +
  theme_axes_math
```
---
layout: true
# Nonstationarity
---
name: problem

## Problem

```{R, gb_all, echo = F, cache = T}
walk_pv <- lapply(X = seq(1, 999, by = 2), FUN = function(i) {
  lm(filter(walk_df, walk == i)$x ~ filter(walk_df, walk == (i+1))$x) %>% summary() %>% coef() %>% extract(2,4)
}) %>% unlist()
```

In our data, `r 100*mean(walk_pv < 0.05)` percent of (independently generated) pairs reject the null hypothesis at the 5% level.

--

**The point?**
--
 If our disturbance is nonstationary, we cannot trust plain OLS.

--

Random walks are only one example of .pink[nonstationary processes]...

.hi[Random walk:] $u_t = u_{t-1} + \varepsilon_t$

--

.hi[Random walk with drift:] $u_t = \alpha_0 + u_{t-1} + \varepsilon_t$

--

.hi[Deterministic trend:] $u_t = \alpha_0 + \beta_1 t + \varepsilon_t$

---
layout: true
# Nonstationarity
## A potential solution
---
name: solution

Some processes are .hi[difference stationary], which means we can get back to our stationarity (good behavior) requirement by taking the difference between $u_t$ and $u_{t-1}$.

--

.hi-slate[Nonstationary:] $u_t = u_{t-1} + \varepsilon_t$ .slate[(a random walk)]
--
<br>.hi[Stationary:] $u_t - u_{t-1} = u_{t-1} + \varepsilon_t - u_{t-1} = \color{#e64173}{\varepsilon_t}$

--

So if we have good reason to believe that our disturbances follow a random walk, we can use OLS on the differences, _i.e._,
--
$$
\begin{align}
  y_t &= \beta_0 + \beta_1 x_t + u_t \\
  y_{t-1} &= \beta_0 + \beta_1 x_{t-1} + u_{t-1} \\
  y_t - y_{t-1} &= \beta_1 \left( x_t - x_{t-1} \right) + \left( u_t - u_{t-1} \right) \\
  \Delta y_t &= \beta_1 \Delta x_t + \Delta u_t
\end{align}
$$
---
name: test
layout: false
# Nonstationarity
## Testing

.pink[Dickey-Fuller] and .pink[augmented Dickey-Fuller] tests are popular ways to test of random walks and other forms of nonstationarity.

--

.hi[Dickey-Fuller tests] compare

H.sub[o]: $y_t = \beta_0 + \beta_1 y_{t-1} + u_t$ with $|\beta_1|<1$ (.hi[stationarity])
<br>
H.sub[a]: $y_t = y_{t-1} + \varepsilon_t$ (.hi[random walk])

--

using a *t* test that $|\beta_1|<1$.<sup>.pink[†]</sup>

.footnote[
.pink[†] People often just test $\beta_1<1$.
]

---
layout: false
# Table of contents

.pull-left[
### Admin
.smallest[

1. [Schedule](#schedule)
]
]

.pull-right[
### Nonstationarity
.smallest[

1. [Introduction](#intro)
1. [Random walks](#walks)
1. [The actual problem](#problem)
1. [A potential solution](#solution)
1. [Dickey-Fuller tests](#test)
]
]