layout | title |
---|---|
post |
Taking the voodoo out of multiple regression |
Valerio Filoso (2013) writes:
Most econometrics textbooks limit themselves to providing the formula for the
$$\beta$$ vector of the type
$$\beta = (X′X)^{-1} X'Y.$$ Although compact and easy to remember, this formulation is a sort black box, since it hardly reveals anything about what really happens during the estimation of a multivariate OLS model. Furthermore, the link between the
$$\beta$$ and the moments of the data distribution disappear buried in the intricacies of matrix algebra. Luckily, an enlightening interpretation of the $$\beta$$s in the multivariate case exists and has relevant interpreting power. It was originally formulated more than seventy years ago by Frisch and Waugh (1933), revived by Lovell (1963), and recently brought to a new life by Angrist and Pischke (2009) under the catchy phrase regression anatomy. According to this result, given a model with K independent variables, the coefficient$$\beta$$ for the k-th variable can be written as
$$ \beta_k = \frac{cov(y,\tilde{x}_k)}{var(\tilde x_k)}$$ where
$$\tilde x_k$$ is the residual obtained by regressing$$x_k$$ on all remaining$$K − 1$$ independent variables.The result is striking since it establishes the possibility of breaking a multivariate model with
$$K$$ independent variables into$$K$$ bivariate models and also sheds light into the machinery of multivariate OLS. This property of OLS does not depend on the underlying Data Generating Process or on its causal interpretation: it is a mechanical property of the estimator which holds because of the algebra behind it.
From,
I'll stick to the first expression in what follows. (See Filoso sections 2-4 for a discussion of the two options. The second is the Frisch-Waugh-Lovell theorem, the first is what Angrist and Pischke call regression anatomy).
Multiple regression with
This is why it's nice that you can break a model with
Similarly, it's possible to arrive at the coefficients of a
Judd et al. (2017) have a nice detailed walk-through of the
I made this is PowerPoint, not knowing how to do it better. Here is the file.
Footnotes
-
The grey ones are redundant and included for ease of notation. ↩