No description, website, or topics provided.
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.
data data for lab 4 Apr 25, 2017
models cookiecutter init Apr 2, 2017
Makefile cookiecutter init Apr 2, 2017

UC Davis Statistics 208 : Statistical Machine Learning

A Course on the Principles of Statistical Machine Learning with Examples in Python

Machine learning is how to get computers to automatically learn and improve with experience. Experience comes in the form of data, improvement is with respect to some performance metric, and learning is done by a learning algorithm. There are always computational constraints, such as the architecture, computation time, bandwidth limitations, and so on. So we can more precisely restate the goal thus: to construct learning algorithms that use data to improve with respect to a performance metric and do so under computational constraints.

We will focus on principles of statistical machine learning in the prediction problems, regression and classification. Conspicuously absent is any Bayesian methodology, hidden Markov models, unsupervised learning including density estimation, clustering, dimension reduction, network modelling, etc. This course is not a broad overview of all of machine learning, but rather a tour of the key ideas in machine learning as told through these prediction tasks. Typically, I have students tell me something along the lines of "I thought machine learning was about [insert random methodology here]". Machine learning is a field, like physical chemistry or creative literature. It is not defined by a couple of methods or a single task, and cannot be taught in a single quarter. With that said, I want this course to lay the foundation for a rich understanding of machine learning.


  1. Class time will be for lectures and labs. These will be posted in advance in their sections. Labs are not graded, and you will have an opportunity to go over your results in class.
  2. Homework will be due every week or so, starting with the second week. (50% of grade)
  3. Discussions and questions should be posted on the slack site: find the channel 2018-sta208
  4. During the quarter, I will try to check my email at the end of the day on weekdays:
  5. A final project will be due at the end of the class. Instructions can be found here. (50% of grade)
  6. You will need to install Python and the necessary packages to participate in this class. See the following instructions for installation notes.

Office Hours

  • Prof. James: 10-11 Tue, 10-11 Thu, 3-4 Fri (MSB 4107)
  • Liwei Wu: 2-4 Wed (Stat TA room)
  • Siteng Hao: 10-12 Fri (Stat TA room)




Introduction to Machine Learning

Principles: Over/under-fitting, training and testing, losses
Reading: ESL Chapter 2

4/2Lecture 1Introduction to machine learning and Python I
4/4Lecture 2Introduction to machine learning and Python II, Lab 1

Regression (beyond Ordinary Least Squares)

Principles: Convex relaxation, computational intractability in subset selection
Reading: ESL Chapter 3, Boyd Chapter 1

4/9Lecture 3OLS, Matrix Decompositions, leave-1-out cross validation
4/11Lecture 4Subset selection and ridge regression, Lab 2, Solutions
4/16Lecture 5Convex optimization
4/18Lecture 6The Lasso, Discussion on HW1


Principles: Surrogate losses, generative and discriminative methods
Reading: ESL Chapter 4

4/23Lecture 7Logistic regression
4/25Lecture 7Classification and Generative Methods, Lab 3, Solns
4/30Lecture 8Max-margin Methods

Unsupervised Learning

Principles: Compression, Dimension Reduction Reading: ESL Chapter 14

5/2Lecture 9Clustering (notes)
5/7Lecture 10Dimension Reduction, Lab 4

Basis Expansion and Kernels

Principles: Feature extraction, the kernel trick, analysis/sythesis duality
Reading: ESL Chapter 5

5/9Lecture 11Basis expansion and hi-di embeddings
5/14Lecture 12Kernels

Resampling, Trees, and Aggregation

Principles: Interpretable models, statistical complexity, learning from experts
Reading: ESL Chapters 7, 8

5/14Lecture 13Bootstrap, Decision Trees, and Random Forests
5/16Lecture 14Boosting

Stochastic Optimization, Neural Networks

Principes: online learning, stochastic optimization, neural networks basics
5/21Lecture 16Stochastic gradient and online learning
5/23Lecture 17Gradient boosting
5/30Lecture 18Neural Networks


6/4Lecture 19Networks and Manifold Learning
6/6Lecture 21Convolutional Nets

Repository Organization

├── Makefile           <- Makefile with commands like `make data` or `make train`
├──          <- The top-level README for developers using this project.
├── data
│   ├── external       <- Data from third party sources.
│   ├── interim        <- Intermediate data that has been transformed.
│   ├── processed      <- The final, canonical data sets for modeling.
│   └── raw            <- The original, immutable data dump.
├── labs               <- notebook files for course labs (6 labs)
├── homeworks          <- notebook files for homeworks (6 hws)
├── lectures           <- notebook files for lectures (naming convention 
├── references         <- Reference material, pdfs, etc.
├── reports            <- Generated analysis as HTML, PDF, LaTeX, etc.
│   └── figures        <- Generated graphics and figures to be used in reporting
├── src                <- Source code for use in this project.
│   ├──    <- Makes src a Python module
│   │
│   ├── data           <- Scripts to download or generate data
│   │   └──
│   │
│   ├── features       <- Scripts to turn raw data into features for modeling
│   │   └──
│   │
│   ├── models         <- Scripts to train models and then use trained models to make
│   │   │                 predictions
│   │   ├──
│   │   └──
│   │
│   └── visualization  <- Scripts to create exploratory and results oriented visualizations
│       └──
└── tox.ini            <- tox file with settings for running tox; see

Project based on the cookiecutter data science project template. #cookiecutterdatascience