Gaussian vs Gaussian Mixture Models on MNIST

Assignment: Maximum Likelihood Estimation of Parametric Models

---

👩‍💻 Team Members

• Aly El-Deen Yasser
• Malak Mohammed
• Ahmed Tarek

---

🧠 Project Overview

This project compares the performance of two parametric probabilistic models — Gaussian (Naive Bayes) and Gaussian Mixture Models (GMMs) — in classifying handwritten digits from the MNIST dataset.

Both models are trained using Maximum Likelihood Estimation (MLE) principles:

• The Gaussian model assumes each class is represented by a single multivariate Gaussian distribution (diagonal covariance).
• The Gaussian Mixture Model (GMM) assumes each class is composed of multiple Gaussian components, estimated using the Expectation-Maximization (EM) algorithm.

We evaluate both models on empirical accuracy and ROC curves for each of the ten digit classes (0–9).

---

📊 Objective

• Apply MLE to estimate parameters of Gaussian and GMM models.
• Use the models to perform classification on the MNIST dataset.
• Compare:
  • Classification accuracy
  • Per-class ROC curves (10 ROCs total)
  • AUC (Area Under Curve) metrics

---

⚙️ Implementation Steps

1. Load and preprocess MNIST dataset

   • Flatten images (28×28 → 784 features)
   • Normalize pixel values to [0,1]
   • Apply PCA (50 dimensions) for numerical stability and speed

2. Train two models

   • Gaussian Naive Bayes (diagonal covariance, MLE estimation)
   • Gaussian Mixture Model (per class, fitted via EM algorithm)

3. Evaluate on test data

   • Compute predicted probabilities and class labels
   • Calculate:
     • Accuracy
     • One-vs-Rest ROC curves for each class
     • AUC per class

4. Compare performance

   • Plot and save ROC curves for both models
   • Summarize numerical results in a report