bin-prior.md

layout	mathjax	author	affiliation	e_mail	date	title	chapter	section	topic	theorem	sources	proof_id	shortcut	username
proof	true	Joram Soch	BCCN Berlin	joram.soch@bccn-berlin.de	2020-01-23 15:38:00 -0800	Conjugate prior distribution for binomial observations	Statistical Models	Count data	Binomial observations	Conjugate prior distribution	authors year title in pages url Wikipedia 2020 Binomial distribution Wikipedia, the free encyclopedia retrieved on 2020-01-23 https://en.wikipedia.org/wiki/Binomial_distribution#Estimation_of_parameters	P29	bin-prior	JoramSoch

Theorem: Let $y$ be the number of successes resulting from $n$ independent trials with unknown success probability $p$, such that $y$ follows a binomial distribution:

$$ \label{eq:Bin} y \sim \mathrm{Bin}(n,p) ; . $$

Then, the conjugate prior for the model parameter $p$ is a beta distribution:

$$ \label{eq:Beta} \mathrm{p}(p) = \mathrm{Bet}(p; \alpha_0, \beta_0) ; . $$

Proof: With the probability mass function of the binomial distribution, the likelihood function implied by \eqref{eq:Bin} is given by

$$ \label{eq:Bin-LF} \mathrm{p}(y|p) = {n \choose y} , p^y , (1-p)^{n-y} ; . $$

In other words, the likelihood function is proportional to a power of $p$ times a power of $(1-p)$:

$$ \label{eq:Bin-LF-prop} \mathrm{p}(y|p) \propto p^y , (1-p)^{n-y} ; . $$

The same is true for a beta distribution over $p$

$$ \label{eq:Bin-prior-s1} \mathrm{p}(p) = \mathrm{Bet}(p; \alpha_0, \beta_0) $$

the probability density function of which

$$ \label{eq:Bin-prior-s2} \mathrm{p}(p) = \frac{1}{B(\alpha_0,\beta_0)} , p^{\alpha_0-1} , (1-p)^{\beta_0-1} $$

exhibits the same proportionality

$$ \label{eq:Bin-prior-s3} \mathrm{p}(p) \propto p^{\alpha_0-1} , (1-p)^{\beta_0-1} $$

and is therefore conjugate relative to the likelihood.