| layout |
mathjax |
author |
affiliation |
e_mail |
date |
title |
chapter |
section |
topic |
theorem |
sources |
proof_id |
shortcut |
username |
proof |
true |
Joram Soch |
BCCN Berlin |
joram.soch@bccn-berlin.de |
2020-02-27 13:16:00 -0800 |
Log family evidences in terms of log model evidences |
Model Selection |
Bayesian model selection |
Family evidence |
Calculation from log model evidences |
| authors |
year |
title |
in |
pages |
url |
doi |
Soch J, Allefeld C |
2018 |
MACS – a new SPM toolbox for model assessment, comparison and selection |
Journal of Neuroscience Methods |
vol. 306, pp. 19-31, eq. 16 |
|
10.1016/j.jneumeth.2018.05.017 |
|
|
P65 |
lfe-lme |
JoramSoch |
Theorem: Let $m_1, \ldots, m_M$ be $M$ statistical models with log model evidences $\mathrm{LME}(m_1), \ldots, \mathrm{LME}(m_M)$ and belonging to $F$ mutually exclusive model families $f_1, \ldots, f_F$. Then, the log family evidences are given by:
$$ \label{eq:LFE-LME}
\mathrm{LFE}(f_j) = \log \sum_{m_i \in f_j} \left[ \exp[\mathrm{LME}(m_i)] \cdot p(m_i|f_j) \right], \quad j = 1, \ldots, F,
$$
where $p(m_i \vert f_j)$ are within-family prior model probabilities.
Proof: Let us consider the (unlogarithmized) family evidence $p(y \vert f_j)$. According to the law of marginal probability, this conditional probability is given by
$$ \label{eq:FE-ME-s1}
p(y|f_j) = \sum_{m_i \in f_j} \left[ p(y|m_i,f_j) \cdot p(m_i|f_j) \right] ; .
$$
Because model families are mutually exclusive, it holds that $p(y \vert m_i,f_j) = p(y \vert m_i)$, such that
$$ \label{eq:FE-ME-s2}
p(y|f_j) = \sum_{m_i \in f_j} \left[ p(y|m_i) \cdot p(m_i|f_j) \right] ; .
$$
Logarithmizing transforms the family evidence $p(y \vert f_j)$ into the log family evidence $\mathrm{LFE}(f_j)$:
$$ \label{eq:LFE-LME-s1}
\mathrm{LFE}(f_j) = \log \sum_{m_i \in f_j} \left[ p(y|m_i) \cdot p(m_i|f_j) \right] ; .
$$
The definition of the log model evidence
$$ \label{eq:LME}
\mathrm{LME}(m) = \log p(y|m)
$$
can be exponentiated to then read
$$ \label{eq:ME}
\exp\left[ \mathrm{LME}(m) \right] = p(y|m)
$$
and applying \eqref{eq:ME} to \eqref{eq:LFE-LME-s1}, we finally have:
$$ \label{eq:LFE-LME-s2}
\mathrm{LFE}(f_j) = \log \sum_{m_i \in f_j} \left[ \exp[\mathrm{LME}(m_i)] \cdot p(m_i|f_j) \right] ; .
$$