Statistical Pattern Recognition

Overview

This repository contains course projects and homework assignments for Statistical Pattern Recognition, focusing on advanced probabilistic modeling and machine learning techniques. Topics include probabilistic inference, Bayesian methods, sampling, clustering, and more.

---

Course Projects

CP1: Word Prediction

• Objective: Build probabilistic models for word prediction.
• Key Topics:
  • Probabilistic language modeling.
  • Word prediction techniques using n-grams and smoothing.

CP2: Bayesian Linear Regression

• Objective: Implement Bayesian linear regression and explore probabilistic inference.
• Key Topics:
  • Posterior distribution analysis.
  • Predictive uncertainty quantification.

CP3: Markov Chain Monte Carlo (MCMC)

• Objective: Develop MCMC algorithms for sampling from complex distributions.
• Key Topics:
  • Gibbs sampling and Metropolis-Hastings.
  • Convergence diagnostics and mixing behavior.

CP4: Coordinate Ascent for Gaussian Mixtures

• Objective: Fit Gaussian Mixture Models (GMMs) using coordinate ascent optimization.
• Key Topics:
  • Expectation-Maximization (EM) algorithm.
  • Coordinate-wise updates for mixture models.

---

Homework Assignments

HW1: Probability Distributions

• Objective: Explore and analyze fundamental probability distributions.
• Key Topics:
  • Discrete and continuous distributions.
  • Maximum likelihood estimation (MLE).

HW2: Gaussian Models

• Objective: Understand Gaussian distributions and their applications.
• Key Topics:
  • Multivariate Gaussian models.
  • Covariance estimation.

HW3: Sampling and Markov Chains

• Objective: Implement sampling methods and study Markov chains.
• Key Topics:
  • Importance sampling and rejection sampling.
  • Markov chain properties and mixing time.

HW4: Clustering Techniques

• Objective: Apply clustering algorithms to real-world datasets.
• Key Topics:
  • k-Means, hierarchical clustering, and GMMs.
  • Evaluating cluster quality using internal metrics.