cat-cov.md

layout	mathjax	author	affiliation	e_mail	date	title	chapter	section	topic	theorem	sources	proof_id	shortcut	username
proof	true	Joram Soch	BCCN Berlin	joram.soch@bccn-berlin.de	2022-09-09 09:57:00 -0700	Covariance matrix of the categorical distribution	Probability Distributions	Multivariate discrete distributions	Categorical distribution	Covariance		P338	cat-cov	JoramSoch

Theorem: Let $X$ be a random vector following a categorical distribution:

$$ \label{eq:cat} X \sim \mathrm{Cat}(n,p) ; . $$

Then, the covariance matrix of $X$ is

$$ \label{eq:cat-cov} \mathrm{Cov}(X) = \mathrm{diag}(p) - pp^\mathrm{T} ; . $$

Proof: The categorical distribution is a special case of the multinomial distribution in which $n = 1$:

$$ \label{eq:cat-mult} X \sim \mathrm{Mult}(n,p) \quad \text{and} \quad n = 1 \quad \Rightarrow \quad X \sim \mathrm{Cat}(p) ; . $$

The covariance matrix of the multinomial distribution is

$$ \label{eq:mult-cov} \mathrm{Cov}(X) = n \left(\mathrm{diag}(p) - pp^\mathrm{T} \right) ; , $$

thus the covariance matrix of the categorical distribution is

$$ \label{eq:cat-cov-qed} \mathrm{Cov}(X) = \mathrm{diag}(p) - pp^\mathrm{T} ; . $$