Estimation of a Random Variable

Estimate $X$ from $Y$ when $(X, Y)$ is bivariate Gaussian, and compare six classical estimators by their empirical MSE on $N=1000$ samples.

#	Information available	Estimator
1	$f_X$ only	Blind — $\hat X = \mu_X$
2	$f_X$ and event $A={X>\mu_Y/2}$	Conditional — $E[X\mid A]$
3	$f_{X,Y}$	Optimal MMSE — $E[X\mid Y]$
4	$f_{X,Y}$	Linear MMSE
5	$f_{X,Y}$	MAP — $\arg\max f_{X\mid Y}$
6	$f_{Y\mid X}$	ML — $\arg\max f_{Y\mid X}$

Pipeline

1. Generate $X \sim \mathcal{N}(0,1)$ via Box-Muller from $U(0,1)$ pairs.
2. Draw $(X, Y)$ from a bivariate Gaussian with sample $\mu_X,\sigma_X^2$ and chosen $\mu_Y=0.01$, $\sigma_Y^2=1.3$, $\rho=0.9$.
3. Compute each estimator and its empirical MSE.

Box-Muller transform

Bivariate Gaussian samples

Results

Blind                MSE = 0.9841
X > μ_Y/2            MSE = 0.3538
Optimal MMSE | Y     MSE = 0.0165
Linear MMSE | Y      MSE = 0.0165
MAP | Y              MSE = 0.0165
ML | Y               MSE = 0.0198

Estimators as a function of $Y$

Distribution of true $X$ vs estimates

Takeaways

• For jointly Gaussian $(X, Y)$: MMSE = MAP = Linear estimator — all equal $E[X\mid Y]$.
• ML discards the prior on $X$, so it is biased and incurs higher MSE.
• Observing $Y$ shrinks MSE by ~60× over the blind estimate at $\rho=0.9$.

Run

python main.py                   # prints all six MSEs
jupyter notebook estimation.ipynb  # notebook with figures