title | date | tags | draft | summary | |||
---|---|---|---|---|---|---|---|
Deriving the OLS Estimator |
2020-12-21 |
|
false |
How to derive the OLS Estimator with matrix notation and a tour of math typesetting using markdown with the help of KaTeX. |
Parsing and display of math equations is included in this blog template. Parsing of math is enabled by remark-math
and rehype-katex
.
KaTeX and its associated font is included in _document.js
so feel free to use it on any page.
1
Inline math symbols can be included by enclosing the term between the $
symbol.
Math code blocks are denoted by $$
.
If you intend to use the $
sign instead of math, you can escape it (\$
), or specify the HTML entity ($
) 2
Inline or manually enumerated footnotes are also supported. Click on the links above to see them in action.
Using matrix notation, let
The vector of outcome variables
\mathbf{Y} = \left[\begin{array}
{c}
y_1 \\
. \\
. \\
. \\
y_n
\end{array}\right]
The matrix of regressors
\mathbf{X} = \left[\begin{array}
{ccccc}
x_{11} & . & . & . & x_{1k} \\
. & . & . & . & . \\
. & . & . & . & . \\
. & . & . & . & . \\
x_{n1} & . & . & . & x_{nn}
\end{array}\right] =
\left[\begin{array}
{c}
\mathbf{x}'_1 \\
. \\
. \\
. \\
\mathbf{x}'_n
\end{array}\right]
The vector of error terms
At times it might be easier to use vector notation. For consistency, I will use the bold small x to denote a vector and capital letters to denote a matrix. Single observations are denoted by the subscript.
Start:
Assumptions:
- Linearity (given above)
-
$E(\mathbf{U}|\mathbf{X}) = 0$ (conditional independence) - rank(
$\mathbf{X}$ ) =$k$ (no multi-collinearity i.e. full rank) -
$Var(\mathbf{U}|\mathbf{X}) = \sigma^2 I_n$ (Homoskedascity)
Aim:
Find
Solution:
Hints:
Take matrix derivative w.r.t
\begin{aligned}
\min Q & = \min_{\beta} \mathbf{Y}'\mathbf{Y} - 2\beta'\mathbf{X}'\mathbf{Y} +
\beta'\mathbf{X}'\mathbf{X}\beta \\
& = \min_{\beta} - 2\beta'\mathbf{X}'\mathbf{Y} + \beta'\mathbf{X}'\mathbf{X}\beta \\
\text{[FOC]}~~~0 & = - 2\mathbf{X}'\mathbf{Y} + 2\mathbf{X}'\mathbf{X}\hat{\beta} \\
\hat{\beta} & = (\mathbf{X}'\mathbf{X})^{-1}\mathbf{X}'\mathbf{Y} \\
& = (\sum^{n} \mathbf{x}_i \mathbf{x}'_i)^{-1} \sum^{n} \mathbf{x}_i y_i
\end{aligned}
Footnotes
-
For the full list of supported TeX functions, check out the KaTeX documentation ↩
-
$10 and $20. ↩