This repository contains the lecture materials for EECS 545, a graduate course in Machine Learning, at the University of Michigan, Ann Arbor.
Jupyter Notebook HTML Python
Switch branches/tags
Nothing to show
Clone or download
Jacob Abernethy
Latest commit 868397d Aug 29, 2016
Permalink
Failed to load latest commit information.
lecture01_introduction renamed folders for readability Jan 25, 2016
lecture02_linear-algebra-optimization A small typo, maybe? Jan 29, 2016
lecture03_convex-functions-optimization Revision to lec07 and Slight Changes to Previous Lectures Jun 30, 2016
lecture04_linear-regression-part1 remove pyc files Jul 12, 2016
lecture05_linear-regression-part2 Edits to Zhe’s work. Waiting to fix the dirichlet issue Jul 12, 2016
lecture06_bayesian-lr-gaussian-processes Edits to Zhe’s work. Waiting to fix the dirichlet issue Jul 12, 2016
lecture07_logistic-regression-perceptron fixing Dirichlet Issus Aug 10, 2016
lecture08_NaiveBayes-GDA-LDA fixing Dirichlet Issus Aug 10, 2016
lecture09_kernel-methods Edits to Zhe’s work. Waiting to fix the dirichlet issue Jul 12, 2016
lecture10_SVM adding pdf file Aug 29, 2016
lecture11_svm-gaussians Revision to Lec08 Jul 3, 2016
lecture12_ml_advice update pdfs 12,13,14 Mar 10, 2016
lecture13_bias-variance-and-info-theory update pdfs 12,13,14 Mar 10, 2016
lecture14_bayesian_networks lecture14: images Mar 15, 2016
lecture15_latent-variables-independence-clustering lecture15: pdf Mar 15, 2016
lecture16_clustering-mixtures-em Improving notation Apr 20, 2016
lecture17_gmm-hmm more fixes to lec 17 Mar 29, 2016
lecture18_pgm-inference-applications Improved clarity of EM for Variational inference Apr 21, 2016
lecture19_pca_ica added edits to lec 19 Apr 3, 2016
lecture20_mds_nldr changes before lecture Mar 30, 2016
lecture21_ml_applications_to_vision adding Matt's lecture on ML and vision Apr 12, 2016
lecture22_neural-networks-part1 giving junhyuk credit for slide prep Apr 12, 2016
lecture23_neural-networks-part2 giving junhyuk credit for slide prep Apr 12, 2016
lecture24_decision_trees_bagging_boosting adding eric lec 25 Apr 19, 2016
lecture25_ML_and_marketing_guest_lecture adding eric lec 25 Apr 19, 2016
tutorials correct typos in tutorial 4 Feb 3, 2016
.gitignore Updating gitignore to ignore pyc files Jul 12, 2016
Lectures.md fixing Dirichlet Issus Aug 10, 2016
README.md made nothing change to readme file May 19, 2016

README.md

EECS 545, Winter 2016

This repository contains the lecture materials for EECS 545, a graduate course in Machine Learning, at the University of Michigan, Ann Arbor.

Formatted Lecture Materials

The link above gives a list of all of the available lecture materials, including links to ipython notebooks (via Jupyter's nbviewer), the slideshow view, and PDFs.

Lecture Readings

We will make references to the following textbooks throughout the course. The only required textbook is Bishop, PRML, but the others are very well-written and offer unique perspectives.

Lecture 01: Introduction to Machine Learning

Wednesday, January 6, 2016

No required reading.

Lecture 02: Linear Algebra & Optimization

Monday, January 11, 2016

  • There are lots of places to look online for linear algebra help!
  • Juan Klopper has a nice online review, based on Jupyter notebooks.

Lecture 03: Convex Functions & Probability

Wednesday, January 13, 2016   (Notebook Viewer, PDF File, Slide Viewer)

Required:

  • Bishop, §1.2: Probability Theory
  • Bishop, §2.1-2.3: Binary, Multinomial, and Normal Random Variables

Optional:

  • Murphy, Chapter 2: Probability

Lecture 04: Linear Regression, Part I

Wednesday, January 20, 2016   (Notebook Viewer, PDF File, Slide Viewer)

Required:

  • Bishop, §1.1: Polynomial Curve Fitting Example
  • Bishop, §3.1: Linear Basis Function Models

Optional:

  • Murphy, Chapter 7: Linear Regression

Lecture 05: Linear Regression, Part II

Monday, January 25, 2016   (Notebook Viewer, PDF File, Slide Viewer)

Required:

  • Bishop, §3.2: The Bias-Variance Decomposition
  • Bishop, §3.3: Bayesian Linear Regression

Optional:

  • Murphy, Chapter 7: Linear Regression

Lecture 06: Probabilistic Models & Logistic Regression

Wednesday, January 27, 2016   (Notebook Viewer, PDF File, Slide Viewer)

Required:

  • Bishop, §4.2: Probabilistic Generative Models
  • Bishop, §4.3: Probabilistic Discriminative Models

Optional:

  • Murphy, Chapter 8: Logistic Regression

Lecture 07: Linear Classifiers

Monday, February 1, 2016   (Notebook Viewer, PDF File, Slide Viewer)

Required:

  • Bishop, §4.1: Discriminant Functions

Recommended:

  • Murphy §3.5: Naive Bayes Classifiers
  • Murphy §4.1: Gaussian Models
  • Murphy §4.2: Gaussian Discriminant Analysis

Optional:

Lecture 08: Kernel Methods I, Kernels

Monday, February 8, 2016

Required:

  • Bishop, §6.1: Dual Representation
  • Bishop, §6.2: Constructing Kernels
  • Bishop, §6.3: Radial Basis Function Networks

Optional:

  • Murphy, §14.2: Kernel Functions

Lecture 09: Kernel Methods II, Duality & Kernel Regression

Wednesday, February 10, 2016

Required:

  • Bishop, §6.1: Dual Representation
  • Bishop, §6.3: Radial Basis Function Networks

Optional:

Lecture 10: Kernel Methods III, Support Vector Machines & Gaussians

Monday, February 15, 2016

Required:

  • Bishop, §7.1: Maximum Margin Classifiers
  • Bishop, §2.3.0-2.3.1: Gaussian Distributions

Optional:

Lecture 11: Kernel Methods III, Bayesian Linear Regression & Gaussian Processes

Wednesday, February 17, 2016

Required:

  • Bishop, §3.3: Bayesian Linear Regression
  • Bishop, §6.4: Gaussian Processes

Recommended:

  • Murphy, §7.6.1-7.6.2: Bayesian Linear Regression
  • Murphy, §4.3: Inference in Joinly Gaussian Distributions

Further Reading:

  • Rasmussen & Williams, Gaussian Processes for Machine Learning. (available free online)

Lecture 12: Machine Learning Advice

Monday, February 22, 2016

No required reading.

Lecture 13: Information Theory & Exponential Families

Monday, March 7, 2016

Required:

  • Bishop, §1.6: Information Theory
  • Bishop, §2.4: The Exponential Family

Recommended:

  • Murphy, §2.8: Information Theory
  • Murphy, §9.2: Exponential Families

Further Reading:

Lecture 14: Probabilistic Graphical Models

Wednesday, March 9, 2016

Required:

  • Bishop, §8.1: Bayesian Networks
  • Bishop, §8.2: Conditional Independence
  • Bishop, §8.3: Markov Random Fields

Recommended:

  • Murphy, §10.1: Directed Graphical Models
  • Murphy, §10.2: Examples of Directed Graphical Models

Lecture 15: Latent Variables, d-Separation, and K-Means

Monday, March 14, 2016

Required:

  • Bishop, §8.2: Conditional Independence
  • Bishop, §9.1: K-Means Clustering

Recommended:

  • Murphy, §10.5: Conditional Independence Properties
  • Murphy, §11.1: Latent Variable Models

Lecture 16: Clustering & Expectation Maximization

Wednesday, March 16, 2016

Required:

  • Lecture Notes, "Expectation Maximization" (see Lecture 16 folder)
  • Bishop, §9.2: Mixtures of Gaussians
  • Bishop, §9.3: An Alternative View of EM
  • Bishop, §9.4: The EM Algorithm in General

Recommended:

  • Murphy, §10.3: Inference in Bayesian Networks
  • Murphy, §10.4: Learning in Bayesian Networks
  • Murphy, §11.2: Mixture Models
  • Murphy, §11.3: Parameter Estimation for Mixture Models
  • Murphy, §11.4: The Expectation Maximization Algorithm

Lecture 17: Markov & Hidden Markov Models

Monday, March 21, 2016

Required:

  • Bishop, §13.1: Markov Models
  • Bishop, §13.2: Hidden Markov Models

Recommended:

  • Murphy, §17.2: Markov Models
  • Murphy, §17.3: Hidden Markov Models
  • Murphy, §17.4: Inference in HMMs
  • Murphy, §17.5: Learning for HMMS

Lecture 18: Inference & Applications of Graphical Models

Monday, March 23, 2016

Required:

  • Bishop, §10.1: Variational Inference
  • Bishop, §11.2: Markov Chain Monte Carlo

Recommended:

  • Murphy, §19.1-4: Markov Random Fields
  • Murphy, §21.2: Variational Inference
  • Murphy, §21.3: The Mean Field Method
  • Murphy, §23.1-4: Monte Carlo Inference
  • Murphy, §24.1-3: Markov Chain Monte Carlo
  • Murphy, §27.3: Latent Dirichlet Allocation

Lecture 19: Principal Components Analysis & ICA

Monday, March 28, 2016

Required:

  • Bishop, §12.1: Principal Components Analysis
  • Bishop, §12.2: Probabilistic PCA
  • Bishop, §12.3: Kernel PCA
  • Bishop, §12.4: Nonlinear Latent Variable Models

Recommended:

  • Murphy, §12.2: Principal Components Analysis
  • Murphy, §12.4: PCA for Categorical Data
  • Murphy, §12.6: Independent Component Analysis