| course |
course_year |
question_number |
tags |
title |
year |
Statistics |
IB |
36 |
|
Paper 2, Section II, H |
2020 |
Consider the general linear model $Y=X \beta^{0}+\varepsilon$ where $X$ is a known $n \times p$ design matrix with $p \geqslant 2, \beta^{0} \in \mathbb{R}^{p}$ is an unknown vector of parameters, and $\varepsilon \in \mathbb{R}^{n}$ is a vector of stochastic errors with $\mathbb{E}\left(\varepsilon_{i}\right)=0, \operatorname{var}\left(\varepsilon_{i}\right)=\sigma^{2}>0$ and $\operatorname{cov}\left(\varepsilon_{i}, \varepsilon_{j}\right)=0$ for all $i, j=1, \ldots, n$ with $i \neq j$. Suppose $X$ has full column rank.
(a) Write down the least squares estimate $\hat{\beta}$ of $\beta^{0}$ and show that it minimises the least squares objective $S(\beta)=|Y-X \beta|^{2}$ over $\beta \in \mathbb{R}^{p}$.
(b) Write down the variance-covariance matrix $\operatorname{cov}(\hat{\beta})$.
(c) Let $\tilde{\beta} \in \mathbb{R}^{p}$ minimise $S(\beta)$ over $\beta \in \mathbb{R}^{p}$ subject to $\beta_{p}=0$. Let $Z$ be the $n \times(p-1)$ submatrix of $X$ that excludes the final column. Write $\operatorname{down} \operatorname{cov}(\tilde{\beta})$.
(d) Let $P$ and $P_{0}$ be $n \times n$ orthogonal projections onto the column spaces of $X$ and $Z$ respectively. Show that for all $u \in \mathbb{R}^{n}, u^{T} P u \geqslant u^{T} P_{0} u$.
(e) Show that for all $x \in \mathbb{R}^{p}$,
$$\operatorname{var}\left(x^{T} \tilde{\beta}\right) \leqslant \operatorname{var}\left(x^{T} \hat{\beta}\right) .$$
[Hint: Argue that $x=X^{T} u$ for some $u \in \mathbb{R}^{n}$.]