added 2 definitions and 3 proofs

D/anova1.md

@@ -0,0 +1,59 @@
+---
+layout: definition
+mathjax: true
+
+author: "Joram Soch"
+affiliation: "BCCN Berlin"
+e_mail: "joram.soch@bccn-berlin.de"
+date: 2022-11-06 10:23:00
+
+title: "One-way analysis of variance"
+chapter: "Statistical Models"
+section: "Univariate normal data"
+topic: "Analysis of variance"
+definition: "One-way ANOVA"
+
+sources:
+  - authors: "Bortz, Jürgen"
+    year: 1977
+    title: "Einfaktorielle Varianzanalyse"
+    in: "Lehrbuch der Statistik. Für Sozialwissenschaftler"
+    pages: "ch. 12.1, pp. 528ff."
+    url: "https://books.google.de/books?id=lNCyBgAAQBAJ"
+  - authors: "Denziloe"
+    year: 2018
+    title: "Derive the distribution of the ANOVA F-statistic under the alternative hypothesis"
+    in: "StackExchange CrossValidated"
+    pages: "retrieved on 2022-11-06"
+    url: "https://stats.stackexchange.com/questions/355594/derive-the-distribution-of-the-anova-f-statistic-under-the-alternative-hypothesi"
+
+def_id: "D181"
+shortcut: "anova1"
+username: "JoramSoch"
+---
+
+
+**Definition:** Consider measurements $y_{ij} \in \mathbb{R}$ from disctinct objects $j = 1, \ldots, n_i$ in separate groups $i = 1, \ldots, k$.
+
+Then, in one-way analysis of variance (ANOVA), these measurements are assumed to come from [normal distributions](/D/norm)
+
+$$ \label{eq:anova1}
+y_{ij} \sim \mathcal{N}(\mu_i, \sigma^2) \quad \text{for all} \quad i = 1, \ldots, k \quad \text{and} \quad j = 1, \dots, n_i
+$$
+
+where
+
+* $\mu_i$ is the [expected value](/D/mean) in group $i$ and
+
+* $\sigma^2$ is the common [variance](/D/var) across groups.
+
+Alternatively, the model may be written as
+
+$$ \label{eq:anova1-alt}
+\begin{split}
+y_{ij} &= \mu_i + \varepsilon_{ij} \\
+\varepsilon_{ij} &\overset{\mathrm{i.i.d.}}{\sim} \mathcal{N}(0, \sigma^2)
+\end{split}
+$$
+
+where $\varepsilon_{ij}$ is the [error term](/D/slr) belonging to observation $j$ in category $i$ and $\varepsilon_{ij}$ are the [independent and identically distributed](/D/iid).

D/anova2.md

@@ -0,0 +1,86 @@
+---
+layout: definition
+mathjax: true
+
+author: "Joram Soch"
+affiliation: "BCCN Berlin"
+e_mail: "joram.soch@bccn-berlin.de"
+date: 2022-11-06 13:41:00
+
+title: "Two-way analysis of variance"
+chapter: "Statistical Models"
+section: "Univariate normal data"
+topic: "Analysis of variance"
+definition: "Two-way ANOVA"
+
+sources:
+  - authors: "Bortz, Jürgen"
+    year: 1977
+    title: "Zwei- und mehrfaktorielle Varianzanalyse"
+    in: "Lehrbuch der Statistik. Für Sozialwissenschaftler"
+    pages: "ch. 12.2, pp. 538ff."
+    url: "https://books.google.de/books?id=lNCyBgAAQBAJ"
+  - authors: "ttd"
+    year: 2021
+    title: "Proof on SSAB/s2~chi2(I-1)(J-1) under the null hypothesis HAB: dij=0 for i=1,...,I and j=1,...,J"
+    in: "StackExchange CrossValidated"
+    pages: "retrieved on 2022-11-06"
+    url: "https://stats.stackexchange.com/questions/545807/proof-on-ss-ab-sigma2-sim-chi2-i-1j-1-under-the-null-hypothesis"
+
+def_id: "D182"
+shortcut: "anova2"
+username: "JoramSoch"
+---
+
+
+**Definition:** Let there be two factors $A$ and $B$ with levels $i = 1, \ldots, a$ and $j = 1, \ldots, b$ that are used to group measurements $y_{ijk} \in \mathbb{R}$ from distinct objects $k = 1, \ldots, n_{ij}$ into $a \cdot b$ categories $(i,j) \in \left\lbrace 1, \ldots, a \right\rbrace \times \left\lbrace 1, \ldots, b \right\rbrace$.
+
+Then, in two-way analysis of variance (ANOVA), these measurements are assumed to come from [normal distributions](/D/norm)
+
+$$ \label{eq:anova2-p1}
+y_{ij} \sim \mathcal{N}(\mu_{ij}, \sigma^2) \quad \text{for all} \quad i = 1, \ldots, a, \quad j = 1, \ldots, b, \quad \text{and} \quad k = 1, \dots, n_{ij}
+$$
+
+with
+
+$$ \label{eq:anova2-p2}
+\mu_{ij} = \mu + \alpha_i + \beta_j + \gamma_{ij}
+$$
+
+where
+
+* $\mu$ is called the "grand mean";
+
+* $\alpha_i$ is the additive "main effect" of the $i$-th level of factor $A$;
+
+* $\beta_j$ is the additive "main effect" of the $j$-th level of factor $B$;
+
+* $\gamma_{ij}$ is the non-additive "interaction effect" of category $(i,j)$;
+
+* $\mu_{ij}$ is the [expected value](/D/mean) in category $(i,j)$; and
+
+* $\sigma^2$ is common [variance](/D/var) across all categories.
+
+Alternatively, the model may be written as
+
+$$ \label{eq:anova2-alt}
+\begin{split}
+y_{ijk} &= \mu + \alpha_i + \beta_j + \gamma_{ij} + \varepsilon_{ijk} \\
+\varepsilon_{ijk} &\sim \mathcal{N}(0, \sigma^2)
+\end{split}
+$$
+
+where $\varepsilon_{ijk}$ is the [error term](/D/slr) corresponding to observation $k$ belonging to the $i$-th level of $A$ and the $j$-th level of $B$.
+
+As the two-way ANOVA model is underdetermined, the parameters of the model are additionally subject to the constraints
+
+$$ \label{eq:anova2-cons}
+\begin{split}
+\sum_{i=1}^{a} w_{ij} \alpha_i &= 0 \quad \text{for all} \quad j = 1, \ldots, b \\
+\sum_{j=1}^{b} w_{ij} \beta_j &= 0 \quad \text{for all} \quad i = 1, \ldots, a \\
+\sum_{i=1}^{a} w_{ij} \gamma_{ij} &= 0 \quad \text{for all} \quad j = 1, \ldots, b \\
+\sum_{j=1}^{b} w_{ij} \gamma_{ij} &= 0 \quad \text{for all} \quad i = 1, \ldots, a
+\end{split}
+$$
+
+where the weights are $w_{ij} = n_{ij}/n$ and the total sample size is $n = \sum_{i=1}^{a} \sum_{j=1}^{b} n_{ij}$.

I/ToC.md

@@ -563,61 +563,68 @@ title: "Table of Contents"
    &emsp;&ensp; 1.2.13. **[Cross-validated log Bayes factor](/P/ugkv-cvlbf)** <br>
    &emsp;&ensp; 1.2.14. **[Expectation of cross-validated log Bayes factor](/P/ugkv-cvlbfmean)** <br>
 
-   1.3. Simple linear regression <br>
-   &emsp;&ensp; 1.3.1. *[Definition](/D/slr)* <br>
-   &emsp;&ensp; 1.3.2. **[Special case of multiple linear regression](/P/slr-mlr)** <br>
-   &emsp;&ensp; 1.3.3. **[Ordinary least squares](/P/slr-ols)** (1) <br>
-   &emsp;&ensp; 1.3.4. **[Ordinary least squares](/P/slr-ols2)** (2) <br>
-   &emsp;&ensp; 1.3.5. **[Expectation of estimates](/P/slr-olsmean)** <br>
-   &emsp;&ensp; 1.3.6. **[Variance of estimates](/P/slr-olsvar)** <br>
-   &emsp;&ensp; 1.3.7. **[Distribution of estimates](/P/slr-olsdist)** <br>
-   &emsp;&ensp; 1.3.8. **[Correlation of estimates](/P/slr-olscorr)** <br>
-   &emsp;&ensp; 1.3.9. **[Effects of mean-centering](/P/slr-meancent)** <br>
-   &emsp;&ensp; 1.3.10. *[Regression line](/D/regline)* <br>
-   &emsp;&ensp; 1.3.11. **[Regression line includes center of mass](/P/slr-comp)** <br>
-   &emsp;&ensp; 1.3.12. **[Projection of data point to regression line](/P/slr-proj)** <br>
-   &emsp;&ensp; 1.3.13. **[Sums of squares](/P/slr-sss)** <br>
-   &emsp;&ensp; 1.3.14. **[Transformation matrices](/P/slr-mat)** <br>
-   &emsp;&ensp; 1.3.15. **[Weighted least squares](/P/slr-wls)** (1) <br>
-   &emsp;&ensp; 1.3.16. **[Weighted least squares](/P/slr-wls2)** (2) <br>
-   &emsp;&ensp; 1.3.17. **[Maximum likelihood estimation](/P/slr-mle)** (1) <br>
-   &emsp;&ensp; 1.3.18. **[Maximum likelihood estimation](/P/slr-mle2)** (2) <br>
-   &emsp;&ensp; 1.3.19. **[Sum of residuals is zero](/P/slr-ressum)** <br>
-   &emsp;&ensp; 1.3.20. **[Correlation with covariate is zero](/P/slr-rescorr)** <br>
-   &emsp;&ensp; 1.3.21. **[Residual variance in terms of sample variance](/P/slr-resvar)** <br>
-   &emsp;&ensp; 1.3.22. **[Correlation coefficient in terms of slope estimate](/P/slr-corr)** <br>
-   &emsp;&ensp; 1.3.23. **[Coefficient of determination in terms of correlation coefficient](/P/slr-rsq)** <br>
-
-   1.4. Multiple linear regression <br>
-   &emsp;&ensp; 1.4.1. *[Definition](/D/mlr)* <br>
-   &emsp;&ensp; 1.4.2. **[Special case of general linear model](/P/mlr-glm)** <br>
-   &emsp;&ensp; 1.4.3. **[Ordinary least squares](/P/mlr-ols)** (1) <br>
-   &emsp;&ensp; 1.4.4. **[Ordinary least squares](/P/mlr-ols2)** (2) <br>
-   &emsp;&ensp; 1.4.5. *[Total sum of squares](/D/tss)* <br>
-   &emsp;&ensp; 1.4.6. *[Explained sum of squares](/D/ess)* <br>
-   &emsp;&ensp; 1.4.7. *[Residual sum of squares](/D/rss)* <br>
-   &emsp;&ensp; 1.4.8. **[Total, explained and residual sum of squares](/P/mlr-pss)** <br>
-   &emsp;&ensp; 1.4.9. *[Estimation matrix](/D/emat)* <br>
-   &emsp;&ensp; 1.4.10. *[Projection matrix](/D/pmat)* <br>
-   &emsp;&ensp; 1.4.11. *[Residual-forming matrix](/D/rfmat)* <br>
-   &emsp;&ensp; 1.4.12. **[Estimation, projection and residual-forming matrix](/P/mlr-mat)** <br>
-   &emsp;&ensp; 1.4.13. **[Idempotence of projection and residual-forming matrix](/P/mlr-idem)** <br>
-   &emsp;&ensp; 1.4.14. **[Weighted least squares](/P/mlr-wls)** (1) <br>
-   &emsp;&ensp; 1.4.15. **[Weighted least squares](/P/mlr-wls2)** (2) <br>
-   &emsp;&ensp; 1.4.16. **[Maximum likelihood estimation](/P/mlr-mle)** <br>
-   &emsp;&ensp; 1.4.17. **[Maximum log-likelihood](/P/mlr-mll)** <br>
-   &emsp;&ensp; 1.4.18. **[Deviance function](/P/mlr-dev)** <br>
-   &emsp;&ensp; 1.4.19. **[Akaike information criterion](/P/mlr-aic)** <br>
-   &emsp;&ensp; 1.4.20. **[Bayesian information criterion](/P/mlr-bic)** <br>
-   &emsp;&ensp; 1.4.21. **[Corrected Akaike information criterion](/P/mlr-aicc)** <br>
-
-   1.5. Bayesian linear regression <br>
-   &emsp;&ensp; 1.5.1. **[Conjugate prior distribution](/P/blr-prior)** <br>
-   &emsp;&ensp; 1.5.2. **[Posterior distribution](/P/blr-post)** <br>
-   &emsp;&ensp; 1.5.3. **[Log model evidence](/P/blr-lme)** <br>
-   &emsp;&ensp; 1.5.4. **[Deviance information criterion](/P/blr-dic)** <br>
-   &emsp;&ensp; 1.5.5. **[Posterior probability of alternative hypothesis](/P/blr-pp)** <br>
-   &emsp;&ensp; 1.5.6. **[Posterior credibility region excluding null hypothesis](/P/blr-pcr)** <br>
+   1.3. Analysis of variance <br>
+   &emsp;&ensp; 1.3.1. *[One-way ANOVA](/D/anova1)* <br>
+   &emsp;&ensp; 1.3.2. **[Ordinary least squares for one-way ANOVA](/P/anova1-ols)** <br>
+   &emsp;&ensp; 1.3.3. **[F-test for main effect in one-way ANOVA](/P/anova1-f)** <br>
+   &emsp;&ensp; 1.3.4. *[Two-way ANOVA](/D/anova2)* <br>
+   &emsp;&ensp; 1.3.5. **[Ordinary least squares for two-way ANOVA](/P/anova2-ols)** <br>
+
+   1.4. Simple linear regression <br>
+   &emsp;&ensp; 1.4.1. *[Definition](/D/slr)* <br>
+   &emsp;&ensp; 1.4.2. **[Special case of multiple linear regression](/P/slr-mlr)** <br>
+   &emsp;&ensp; 1.4.3. **[Ordinary least squares](/P/slr-ols)** (1) <br>
+   &emsp;&ensp; 1.4.4. **[Ordinary least squares](/P/slr-ols2)** (2) <br>
+   &emsp;&ensp; 1.4.5. **[Expectation of estimates](/P/slr-olsmean)** <br>
+   &emsp;&ensp; 1.4.6. **[Variance of estimates](/P/slr-olsvar)** <br>
+   &emsp;&ensp; 1.4.7. **[Distribution of estimates](/P/slr-olsdist)** <br>
+   &emsp;&ensp; 1.4.8. **[Correlation of estimates](/P/slr-olscorr)** <br>
+   &emsp;&ensp; 1.4.9. **[Effects of mean-centering](/P/slr-meancent)** <br>
+   &emsp;&ensp; 1.4.10. *[Regression line](/D/regline)* <br>
+   &emsp;&ensp; 1.4.11. **[Regression line includes center of mass](/P/slr-comp)** <br>
+   &emsp;&ensp; 1.4.12. **[Projection of data point to regression line](/P/slr-proj)** <br>
+   &emsp;&ensp; 1.4.13. **[Sums of squares](/P/slr-sss)** <br>
+   &emsp;&ensp; 1.4.14. **[Transformation matrices](/P/slr-mat)** <br>
+   &emsp;&ensp; 1.4.15. **[Weighted least squares](/P/slr-wls)** (1) <br>
+   &emsp;&ensp; 1.4.16. **[Weighted least squares](/P/slr-wls2)** (2) <br>
+   &emsp;&ensp; 1.4.17. **[Maximum likelihood estimation](/P/slr-mle)** (1) <br>
+   &emsp;&ensp; 1.4.18. **[Maximum likelihood estimation](/P/slr-mle2)** (2) <br>
+   &emsp;&ensp; 1.4.19. **[Sum of residuals is zero](/P/slr-ressum)** <br>
+   &emsp;&ensp; 1.4.20. **[Correlation with covariate is zero](/P/slr-rescorr)** <br>
+   &emsp;&ensp; 1.4.21. **[Residual variance in terms of sample variance](/P/slr-resvar)** <br>
+   &emsp;&ensp; 1.4.22. **[Correlation coefficient in terms of slope estimate](/P/slr-corr)** <br>
+   &emsp;&ensp; 1.4.23. **[Coefficient of determination in terms of correlation coefficient](/P/slr-rsq)** <br>
+
+   1.5. Multiple linear regression <br>
+   &emsp;&ensp; 1.5.1. *[Definition](/D/mlr)* <br>
+   &emsp;&ensp; 1.5.2. **[Special case of general linear model](/P/mlr-glm)** <br>
+   &emsp;&ensp; 1.5.3. **[Ordinary least squares](/P/mlr-ols)** (1) <br>
+   &emsp;&ensp; 1.5.4. **[Ordinary least squares](/P/mlr-ols2)** (2) <br>
+   &emsp;&ensp; 1.5.5. *[Total sum of squares](/D/tss)* <br>
+   &emsp;&ensp; 1.5.6. *[Explained sum of squares](/D/ess)* <br>
+   &emsp;&ensp; 1.5.7. *[Residual sum of squares](/D/rss)* <br>
+   &emsp;&ensp; 1.5.8. **[Total, explained and residual sum of squares](/P/mlr-pss)** <br>
+   &emsp;&ensp; 1.5.9. *[Estimation matrix](/D/emat)* <br>
+   &emsp;&ensp; 1.5.10. *[Projection matrix](/D/pmat)* <br>
+   &emsp;&ensp; 1.5.11. *[Residual-forming matrix](/D/rfmat)* <br>
+   &emsp;&ensp; 1.5.12. **[Estimation, projection and residual-forming matrix](/P/mlr-mat)** <br>
+   &emsp;&ensp; 1.5.13. **[Idempotence of projection and residual-forming matrix](/P/mlr-idem)** <br>
+   &emsp;&ensp; 1.5.14. **[Weighted least squares](/P/mlr-wls)** (1) <br>
+   &emsp;&ensp; 1.5.15. **[Weighted least squares](/P/mlr-wls2)** (2) <br>
+   &emsp;&ensp; 1.5.16. **[Maximum likelihood estimation](/P/mlr-mle)** <br>
+   &emsp;&ensp; 1.5.17. **[Maximum log-likelihood](/P/mlr-mll)** <br>
+   &emsp;&ensp; 1.5.18. **[Deviance function](/P/mlr-dev)** <br>
+   &emsp;&ensp; 1.5.19. **[Akaike information criterion](/P/mlr-aic)** <br>
+   &emsp;&ensp; 1.5.20. **[Bayesian information criterion](/P/mlr-bic)** <br>
+   &emsp;&ensp; 1.5.21. **[Corrected Akaike information criterion](/P/mlr-aicc)** <br>
+
+   1.6. Bayesian linear regression <br>
+   &emsp;&ensp; 1.6.1. **[Conjugate prior distribution](/P/blr-prior)** <br>
+   &emsp;&ensp; 1.6.2. **[Posterior distribution](/P/blr-post)** <br>
+   &emsp;&ensp; 1.6.3. **[Log model evidence](/P/blr-lme)** <br>
+   &emsp;&ensp; 1.6.4. **[Deviance information criterion](/P/blr-dic)** <br>
+   &emsp;&ensp; 1.6.5. **[Posterior probability of alternative hypothesis](/P/blr-pp)** <br>
+   &emsp;&ensp; 1.6.6. **[Posterior credibility region excluding null hypothesis](/P/blr-pcr)** <br>
 
 2. Multivariate normal data
 
@@ -773,4 +780,4 @@ title: "Table of Contents"
    3.5. Bayesian model averaging <br>
    &emsp;&ensp; 3.5.1. *[Definition](/D/bma)* <br>
    &emsp;&ensp; 3.5.2. **[Derivation](/P/bma-der)** <br>
-   &emsp;&ensp; 3.5.3. **[Calculation from log model evidences](/P/bma-lme)** <br>
+   &emsp;&ensp; 3.5.3. **[Calculation from log model evidences](/P/bma-lme)** <br>