diff --git a/I/ToC.md b/I/ToC.md index f803fe89..bdb86730 100644 --- a/I/ToC.md +++ b/I/ToC.md @@ -550,25 +550,26 @@ title: "Table of Contents" 1.4. Multiple linear regression
   1.4.1. *[Definition](/D/mlr)*
-    1.4.2. **[Ordinary least squares](/P/mlr-ols)** (1)
-    1.4.3. **[Ordinary least squares](/P/mlr-ols2)** (2)
-    1.4.4. *[Total sum of squares](/D/tss)*
-    1.4.5. *[Explained sum of squares](/D/ess)*
-    1.4.6. *[Residual sum of squares](/D/rss)*
-    1.4.7. **[Total, explained and residual sum of squares](/P/mlr-pss)**
-    1.4.8. *[Estimation matrix](/D/emat)*
-    1.4.9. *[Projection matrix](/D/pmat)*
-    1.4.10. *[Residual-forming matrix](/D/rfmat)*
-    1.4.11. **[Estimation, projection and residual-forming matrix](/P/mlr-mat)**
-    1.4.12. **[Idempotence of projection and residual-forming matrix](/P/mlr-idem)**
-    1.4.13. **[Weighted least squares](/P/mlr-wls)** (1)
-    1.4.14. **[Weighted least squares](/P/mlr-wls2)** (2)
-    1.4.15. **[Maximum likelihood estimation](/P/mlr-mle)**
-    1.4.16. **[Maximum log-likelihood](/P/mlr-mll)**
-    1.4.17. **[Deviance function](/P/mlr-dev)**
-    1.4.18. **[Akaike information criterion](/P/mlr-aic)**
-    1.4.19. **[Bayesian information criterion](/P/mlr-bic)**
-    1.4.20. **[Corrected Akaike information criterion](/P/mlr-aicc)**
+    1.4.2. **[Special case of general linear model](/P/mlr-glm)**
+    1.4.3. **[Ordinary least squares](/P/mlr-ols)** (1)
+    1.4.4. **[Ordinary least squares](/P/mlr-ols2)** (2)
+    1.4.5. *[Total sum of squares](/D/tss)*
+    1.4.6. *[Explained sum of squares](/D/ess)*
+    1.4.7. *[Residual sum of squares](/D/rss)*
+    1.4.8. **[Total, explained and residual sum of squares](/P/mlr-pss)**
+    1.4.9. *[Estimation matrix](/D/emat)*
+    1.4.10. *[Projection matrix](/D/pmat)*
+    1.4.11. *[Residual-forming matrix](/D/rfmat)*
+    1.4.12. **[Estimation, projection and residual-forming matrix](/P/mlr-mat)**
+    1.4.13. **[Idempotence of projection and residual-forming matrix](/P/mlr-idem)**
+    1.4.14. **[Weighted least squares](/P/mlr-wls)** (1)
+    1.4.15. **[Weighted least squares](/P/mlr-wls2)** (2)
+    1.4.16. **[Maximum likelihood estimation](/P/mlr-mle)**
+    1.4.17. **[Maximum log-likelihood](/P/mlr-mll)**
+    1.4.18. **[Deviance function](/P/mlr-dev)**
+    1.4.19. **[Akaike information criterion](/P/mlr-aic)**
+    1.4.20. **[Bayesian information criterion](/P/mlr-bic)**
+    1.4.21. **[Corrected Akaike information criterion](/P/mlr-aicc)**
1.5. Bayesian linear regression
   1.5.1. **[Conjugate prior distribution](/P/blr-prior)**
diff --git a/P/mlr-glm.md b/P/mlr-glm.md new file mode 100644 index 00000000..d59fa37b --- /dev/null +++ b/P/mlr-glm.md @@ -0,0 +1,66 @@ +--- +layout: proof +mathjax: true + +author: "Joram Soch" +affiliation: "BCCN Berlin" +e_mail: "joram.soch@bccn-berlin.de" +date: 2022-07-21 08:28:00 + +title: "Multiple linear regression is a special case of the general linear model" +chapter: "Statistical Models" +section: "Univariate normal data" +topic: "Multiple linear regression" +theorem: "Special case of general linear model" + +sources: + - authors: "Wikipedia" + year: 2022 + title: "General linear model" + in: "Wikipedia, the free encyclopedia" + pages: "retrieved on 2022-07-21" + url: "https://en.wikipedia.org/wiki/General_linear_model#Comparison_to_multiple_linear_regression" + +proof_id: "P329" +shortcut: "mlr-glm" +username: "JoramSoch" +--- + + +**Theorem:** [Multiple linear regression](/D/mlr) is a special case of the [general linear model](/D/mlr) with number of measurements $v = 1$, such that data matrix $Y$, regression coefficients $B$, noise matrix $E$ and noise covariance $\Sigma$ equate as + +$$ \label{eq:mlr-glm} +Y = y, \quad B = \beta, \quad E = \varepsilon \quad \text{and} \quad \Sigma = \sigma^2 +$$ + +where $y$, $\beta$, $\varepsilon$ and $\sigma^2$ are the data vector, regression coefficients, noise vector and noise variance from [multiple linear regression](/D/mlr). + + +**Proof:** The [linear regression model with correlated errors](/D/mlr) is given by: + +$$ \label{eq:mlr} +y = X\beta + \varepsilon, \; \varepsilon \sim \mathcal{N}(0, \sigma^2 V) \; . +$$ + +Because $\varepsilon$ is an $n \times 1$ vector and $\sigma^2$ is scalar, we have the following identities: + +$$ +\begin{split} +\mathrm{vec}(\varepsilon) &= \varepsilon \\ +\sigma^2 \otimes V &= \sigma^2 V \; . +\end{split} +$$ + +Thus, using the [relationship between multivariate normal and matrix normal distribution](/P/matn-mvn), equation \eqref{eq:mlr} can also be written as + +$$ \label{eq:mlr-dev} +y = X\beta + \varepsilon, \; \varepsilon \sim \mathcal{MN}(0, V, \sigma^2) \; . +$$ + +Comparing with the [general linear model with correlated observations](/D/glm) + +$$ \label{eq:glm} +Y = X B + E, \; E \sim \mathcal{MN}(0, V, \Sigma) \; , +$$ + +we finally note the equivalences given in equation \eqref{eq:mlr-glm}. \ No newline at end of file