added 3 definitions and 6 proofs

D/cfm.md

@@ -0,0 +1,43 @@
+---
+layout: definition
+mathjax: true
+
+author: "Joram Soch"
+affiliation: "BCCN Berlin"
+e_mail: "joram.soch@bccn-berlin.de"
+date: 2021-10-21 17:01
+
+title: "General linear model"
+chapter: "Statistical Models"
+section: "Multivariate normal data"
+topic: "Inverse general linear model"
+definition: "Corresponding forward model"
+
+sources:
+  - authors: "Haufe S, Meinecke F, Görgen K, Dähne S, Haynes JD, Blankertz B, Bießmann F"
+    year: 2014
+    title: "On the interpretation of weight vectors of linear models in multivariate neuroimaging"
+    in: "NeuroImage"
+    pages: "vol. 87, pp. 96–110, eq. 3"
+    url: "https://www.sciencedirect.com/science/article/pii/S1053811913010914"
+    doi: "10.1016/j.neuroimage.2013.10.067"
+
+def_id: "D162"
+shortcut: "cfm"
+username: "JoramSoch"
+---
+
+
+**Definition:** Let there be observations $Y \in \mathbb{R}^{n \times v}$ and $X \in \mathbb{R}^{n \times p}$ and consider a weight matrix $W = f(Y,X) \in \mathbb{R}^{v \times p}$ estimated from $Y$ and $X$, such that right-multiplying $Y$ with the weight matrix gives an estimate or prediction of $X$:
+
+$$ \label{eq:bda}
+\hat{X} = Y W \; .
+$$
+
+Given that the columns of $\hat{X}$ are linearly independent, then
+
+$$ \label{eq:cfm}
+Y = \hat{X} A^\mathrm{T} + E \quad \text{with} \quad \hat{X}^\mathrm{T} E = 0
+$$
+
+is called the corresponding forward model relative to the weight matrix $W$.

D/iglm.md

@@ -0,0 +1,37 @@
+---
+layout: definition
+mathjax: true
+
+author: "Joram Soch"
+affiliation: "BCCN Berlin"
+e_mail: "joram.soch@bccn-berlin.de"
+date: 2021-10-21 15:31:00
+
+title: "General linear model"
+chapter: "Statistical Models"
+section: "Multivariate normal data"
+topic: "Inverse general linear model"
+definition: "Definition"
+
+sources:
+  - authors: "Soch J, Allefeld C, Haynes JD"
+    year: 2020
+    title: "Inverse transformed encoding models – a solution to the problem of correlated trial-by-trial parameter estimates in fMRI decoding"
+    in: "NeuroImage"
+    pages: "vol. 209, art. 116449, Appendix C"
+    url: "https://www.sciencedirect.com/science/article/pii/S1053811919310407"
+    doi: "10.1016/j.neuroimage.2019.116449"
+
+def_id: "D161"
+shortcut: "iglm"
+username: "JoramSoch"
+---
+
+
+**Definition:** Let there be a [general linear models](/D/glm) of measured data $Y \in \mathbb{R}^{n \times v}$ in terms of the [design matrix](/D/glm) $X \in \mathbb{R}^{n \times p}$:
+
+$$ \label{eq:glm}
+Y = X B + E, \; E \sim \mathcal{MN}(0, V, \Sigma) \; .
+$$
+
+Then, a [linear model](/D/glm) of $X$ in terms of $Y$, under the assumption of \eqref{eq:glm}, is called an inverse general linear model.

D/tglm.md

@@ -0,0 +1,49 @@
+---
+layout: definition
+mathjax: true
+
+author: "Joram Soch"
+affiliation: "BCCN Berlin"
+e_mail: "joram.soch@bccn-berlin.de"
+date: 2021-10-21 14:43:00
+
+title: "General linear model"
+chapter: "Statistical Models"
+section: "Multivariate normal data"
+topic: "Transformed general linear model"
+definition: "Definition"
+
+sources:
+  - authors: "Soch J, Allefeld C, Haynes JD"
+    year: 2020
+    title: "Inverse transformed encoding models – a solution to the problem of correlated trial-by-trial parameter estimates in fMRI decoding"
+    in: "NeuroImage"
+    pages: "vol. 209, art. 116449, Appendix A"
+    url: "https://www.sciencedirect.com/science/article/pii/S1053811919310407"
+    doi: "10.1016/j.neuroimage.2019.116449"
+
+def_id: "D160"
+shortcut: "tglm"
+username: "JoramSoch"
+---
+
+
+**Definition:** Let there be two [general linear models](/D/glm) of measured data $Y \in \mathbb{R}^{n \times v}$ using [design matrices](/D/glm) $X \in \mathbb{R}^{n \times p}$ and $X_t \in \mathbb{R}^{n \times t}$
+
+$$ \label{eq:glm1}
+Y = X B + E, \; E \sim \mathcal{MN}(0, V, \Sigma)
+$$
+
+$$ \label{eq:glm2}
+Y = X_t \Gamma + E_t, \; E_t \sim \mathcal{MN}(0, V, \Sigma_t)
+$$
+
+and assume that $X_t$ can be transformed into $X$ using a transformation matrix $T \in \mathbb{R}^{t \times p}$
+
+$$ \label{eq:X-Xt-T}
+X = X_t \, T
+$$
+
+where $p < t$ and $X$, $X_t$ and $T$ have full ranks $\mathrm{rk}(X) = p$, $\mathrm{rk}(X_t) = t$ and $\mathrm{rk}(T) = p$.
+
+Then, a [linear model](/D/glm) of the parameter estimates from \eqref{eq:glm2}, under the assumption of \eqref{eq:glm1}, is called a transformed general linear model.

I/Table_of_Contents.md

@@ -512,10 +512,23 @@ title: "Table of Contents"
    &emsp;&ensp; 2.1.3. **[Weighted least squares](/P/glm-wls)** <br>
    &emsp;&ensp; 2.1.4. **[Maximum likelihood estimation](/P/glm-mle)** <br>
 
-   2.2. Multivariate Bayesian linear regression <br>
-   &emsp;&ensp; 2.2.1. **[Conjugate prior distribution](/P/mblr-prior)** <br>
-   &emsp;&ensp; 2.2.2. **[Posterior distribution](/P/mblr-post)** <br>
-   &emsp;&ensp; 2.2.3. **[Log model evidence](/P/mblr-lme)** <br>
+   2.2. Transformed general linear model <br>
+   &emsp;&ensp; 2.2.1. *[Definition](/D/tglm)* <br>
+   &emsp;&ensp; 2.2.2. **[Derivation of the distribution](/P/tglm-dist)** <br>
+   &emsp;&ensp; 2.2.3. **[Equivalence of parameter estimates](/P/tglm-para)** <br>
+
+   2.3. Inverse general linear model <br>
+   &emsp;&ensp; 2.3.1. *[Definition](/D/iglm)* <br>
+   &emsp;&ensp; 2.3.2. **[Derivation of the distribution](/P/iglm-dist)** <br>
+   &emsp;&ensp; 2.3.3. **[Best linear unbiased estimator](/P/iglm-blue)** <br>
+   &emsp;&ensp; 2.3.4. *[Corresponding forward model](/D/cfm)* <br>
+   &emsp;&ensp; 2.3.5. **[Derivation of parameters](/P/cfm-para)** <br>
+   &emsp;&ensp; 2.3.6. **[Proof of existence](/P/cfm-exist)** <br>
+
+   2.4. Multivariate Bayesian linear regression <br>
+   &emsp;&ensp; 2.4.1. **[Conjugate prior distribution](/P/mblr-prior)** <br>
+   &emsp;&ensp; 2.4.2. **[Posterior distribution](/P/mblr-post)** <br>
+   &emsp;&ensp; 2.4.3. **[Log model evidence](/P/mblr-lme)** <br>
 
 3. Poisson data
 

P/cfm-exist.md

@@ -0,0 +1,71 @@
+---
+layout: proof
+mathjax: true
+
+author: "Joram Soch"
+affiliation: "BCCN Berlin"
+e_mail: "joram.soch@bccn-berlin.de"
+date: 2021-10-21 17:43:00
+
+title: "Existence of the corresponding forward model"
+chapter: "Statistical Models"
+section: "Multivariate normal data"
+topic: "Inverse general linear model"
+theorem: "Proof of existence"
+
+sources:
+  - authors: "Haufe S, Meinecke F, Görgen K, Dähne S, Haynes JD, Blankertz B, Bießmann F"
+    year: 2014
+    title: "On the interpretation of weight vectors of linear models in multivariate neuroimaging"
+    in: "NeuroImage"
+    pages: "vol. 87, pp. 96–110, Appendix B"
+    url: "https://www.sciencedirect.com/science/article/pii/S1053811913010914"
+    doi: "10.1016/j.neuroimage.2013.10.067"
+
+proof_id: "P270"
+shortcut: "cfm-exist"
+username: "JoramSoch"
+---
+
+
+**Theorem:** Let there be observations $Y \in \mathbb{R}^{n \times v}$ and $X \in \mathbb{R}^{n \times p}$ and consider a weight matrix $W \in \mathbb{R}^{v \times p}$ predicting $X$ from $Y$:
+
+$$ \label{eq:bda}
+\hat{X} = Y W \; .
+$$
+
+Then, there exists a [corresponding forward model](/D/cfm).
+
+
+**Proof:** The [corresponding forward model](/D/cfm) is defined as
+
+$$ \label{eq:cfm}
+Y = \hat{X} A^\mathrm{T} + E \quad \text{with} \quad \hat{X}^\mathrm{T} E = 0
+$$
+
+and the [parameters of the corresponding forward model](/P/cfm-para) are equal to
+
+$$ \label{eq:cfm-para}
+A = \Sigma_y W \Sigma_x^{-1} \quad \text{where} \quad \Sigma_x = \hat{X}^\mathrm{T} \hat{X} \quad \text{and} \quad \Sigma_y = Y^\mathrm{T} Y \; .
+$$
+
+<br>
+1) Because the columns of $\hat{X}$ are assumed to be linearly independent [by definition of the corresponding forward model](/D/cfm), the matrix $\Sigma_x = \hat{X}^\mathrm{T} \hat{X}$ is invertible, such that $A$ in \eqref{eq:cfm-para} is well-defined.
+
+<br>
+2) Moreover, the solution for the matrix $A$ satisfies the [constraint of the corresponding forward model](/D/cfm) for predicted $X$ and errors $E$ to be uncorrelated which can be shown as follows:
+
+$$ \label{eq:X-E-0}
+\begin{split}
+\hat{X}^\mathrm{T} E &\overset{\eqref{eq:cfm}}{=} \hat{X}^\mathrm{T} \left( Y - \hat{X} A^\mathrm{T} \right) \\
+&\overset{\eqref{eq:cfm-para}}{=} \hat{X}^\mathrm{T} \left( Y - \hat{X} \, \Sigma_x^{-1} W^\mathrm{T} \Sigma_y \right) \\
+&= \hat{X}^\mathrm{T} Y - \hat{X}^\mathrm{T} \hat{X} \, \Sigma_x^{-1} W^\mathrm{T} \Sigma_y \\
+&\overset{\eqref{eq:cfm-para}}{=} \hat{X}^\mathrm{T} Y - \hat{X}^\mathrm{T} \hat{X} \left( \hat{X}^\mathrm{T} \hat{X} \right)^{-1} W^\mathrm{T} \left( Y^\mathrm{T} Y \right) \\
+% &= \hat{X}^\mathrm{T} Y - W^\mathrm{T} \left( Y^\mathrm{T} Y \right) \\
+&\overset{\eqref{eq:bda}}{=} (Y W)^\mathrm{T} Y - W^\mathrm{T} \left( Y^\mathrm{T} Y \right) \\
+&= W^\mathrm{T} Y^\mathrm{T} Y - W^\mathrm{T} Y^\mathrm{T} Y \\
+&= 0 \; .
+\end{split}
+$$
+
+This completes the proof.

P/cfm-para.md

@@ -0,0 +1,79 @@
+---
+layout: proof
+mathjax: true
+
+author: "Joram Soch"
+affiliation: "BCCN Berlin"
+e_mail: "joram.soch@bccn-berlin.de"
+date: 2021-10-21 17:20:00
+
+title: "Parameters of the corresponding forward model"
+chapter: "Statistical Models"
+section: "Multivariate normal data"
+topic: "Inverse general linear model"
+theorem: "Derivation of parameters"
+
+sources:
+  - authors: "Haufe S, Meinecke F, Görgen K, Dähne S, Haynes JD, Blankertz B, Bießmann F"
+    year: 2014
+    title: "On the interpretation of weight vectors of linear models in multivariate neuroimaging"
+    in: "NeuroImage"
+    pages: "vol. 87, pp. 96–110, Theorem 1"
+    url: "https://www.sciencedirect.com/science/article/pii/S1053811913010914"
+    doi: "10.1016/j.neuroimage.2013.10.067"
+
+proof_id: "P269"
+shortcut: "cfm-para"
+username: "JoramSoch"
+---
+
+
+**Theorem:** Let there be observations $Y \in \mathbb{R}^{n \times v}$ and $X \in \mathbb{R}^{n \times p}$ and consider a weight matrix $W \in \mathbb{R}^{v \times p}$ predicting $X$ from $Y$:
+
+$$ \label{eq:bda}
+\hat{X} = Y W \; .
+$$
+
+Then, the parameter matrix of the [corresponding forward model](/D/cfm) is equal to
+
+$$ \label{eq:cfm-para}
+A = \Sigma_y W \Sigma_x^{-1}
+$$
+
+with the [sample covariance](/D/cov-samp)
+
+$$ \label{eq:Sx-Sy}
+\begin{split}
+\Sigma_x &= \hat{X}^\mathrm{T} \hat{X} \\
+\Sigma_y &= Y^\mathrm{T} Y \; .
+\end{split}
+$$
+
+
+**Proof:** The [corresponding forward model](/D/cfm) is given by
+
+$$ \label{eq:cfm}
+Y = \hat{X} A^\mathrm{T} + E \; ,
+$$
+
+subject to the constraint that predicted $X$ and errors $E$ are uncorrelated:
+
+$$ \label{eq:cfm-con}
+\hat{X}^\mathrm{T} E = 0 \; .
+$$
+
+With that, we can directly derive the parameter matrix $A$:
+
+$$ \label{eq:cfm-para-qed}
+\begin{split}
+Y &\overset{\eqref{eq:cfm}}{=} \hat{X} A^\mathrm{T} + E \\
+\hat{X} A^\mathrm{T} &= Y - E \\
+\hat{X}^\mathrm{T} \hat{X} A^\mathrm{T} &= \hat{X}^\mathrm{T} (Y - E) \\
+\hat{X}^\mathrm{T} \hat{X} A^\mathrm{T} &= \hat{X}^\mathrm{T} Y - \hat{X}^\mathrm{T} E \\
+\hat{X}^\mathrm{T} \hat{X} A^\mathrm{T} &\overset{\eqref{eq:cfm-con}}{=} \hat{X}^\mathrm{T} Y \\
+\hat{X}^\mathrm{T} \hat{X} A^\mathrm{T} &\overset{\eqref{eq:bda}}{=} W^\mathrm{T} Y^\mathrm{T} Y \\
+\Sigma_x A^\mathrm{T} &\overset{\eqref{eq:Sx-Sy}}{=} W^\mathrm{T} \Sigma_y \\
+A^\mathrm{T} &= \Sigma_x^{-1} W^\mathrm{T} \Sigma_y \\
+A &= \Sigma_y W \Sigma_x^{-1} \; .
+\end{split}
+$$