This course will cover probabilistic graphical models -- powerful and interpretable models for reasoning under uncertainty. The generic families of models such as directed, undirected, and factor graphs as well as specific representations such as hidden Markov models and conditional random fields will be discussed. The discussions will include both the theoretical aspects of representation, learning, and inference, and their applications in many interesting fields such as computer vision, natural language processing, computational biology, and medical diagnosis.
Important note: This course will focus primiarly on the foundations of probabilistic graphical models (PGM). Almost all of the lecture time will be used for discussing and using probability theory, statistics, optimization, and calculus for understanding, analysis, and derivation of PGM approaches and algorithms. Programming will be minimal.
Strong background in calculus, statistics, and probability.
Recommended: Probabilistic Graphical Models By Daphne Koller and Nir Friedman
https://mitpress.mit.edu/books/probabilistic-graphical-models
Unedited, rough class notes. Not meant to replace your own note taking.
https://1drv.ms/u/s!AjipUklg3ofWkQYm2sHRh9tmf1Wq
- Chapter 1 - Introduction
- Chapter 2 - Foundations
- Chapter 3 - The Bayesian Network Representation
- Chapter 4 - Undirected Graphical Models
- Chapter 5 - Local Probabilistic Models
- Chapter 6 - Template-Based Representations
- Chapter 7 - Gaussian Network Models
- Chapter 8 - The Exponential Family
- Chapter 9 - Variable Elimination
- Chapter 10 - Clique Trees
- Chapter 11 - Inference as Optimization
- Chapter 12 - Particle-Based Approximate Inference
- Chapter 13 - MAP Inference
- Chapter 15 - Inference in Temporal Models
- Chapter 16 - Learning Graphical Models: Overview
- Chapter 17 - Parameter Estimation
- Chapter 18 - Structure Learning in Bayesian Networks
- Chapter 20 - Learning Undirected Models
- Chapter 21 - Causality
- Chapter 22 - Utilities and Decisions
- Chapter 23 - Structured Decision Problems
Mustafa Bilgic
Office hours: Tuesdays, 11am - 12pm
Web: http://www.cs.iit.edu/~mbilgic/
Eadhunath Venghatesan
Office hours: Thursdays, 11am - 12pm
Office: Stuart Building, Room 019
| Item | Points |
|---|---|
| Assignments | 28% |
| Midterm | 32% |
| Final | 40% |
| Total | 100 |
There will be approximately 7 assignments. Assignments will focus on the theory and foundations of probabilistic graphical models. There might be one programming assignment in Python.
There is a 5-minute grace period. After the grace period is over, every late minute costs 1 point. The late submission policy is strictly enforced. Please do not submit late; submit early.
I do not extend deadlines.
I do not assign extra-credit work. Why? For at least two reasons:
- Extra credit is rarely truly optional; every student feels they have to do it, and hence it becomes a required credit in practice.
- The TAs are often overworked and underpaid.
I take the Code of Academic Honesty very seriosly and I report violations to the university. Please read the Code of Academic Honesty in the student handbook: https://web.iit.edu/student-affairs/handbook/fine-print/code-academic-honesty.
Reasonable accommodations (https://web.iit.edu/cdr/services/reasonable-accommodations) are available to the students with documented disabilities. Students must first obtain a letter of accommodation from the Center for Disability Resources: https://web.iit.edu/cdr/