DSCI 572: Supervised Learning II
An introduction to optimization for machine learning. Computation of derivatives. Deep learning.
2019 Instructor: Mike Gelbart
2019 Lecture Schedule
Course structure: this course will be delivered as a flipped classroom, which means you are expected to watch the lecture videos before class. While you're not expected to understand everything just from watching a video once, you're expected to come to class with a basic familiarity of what it's all about; we won't be starting again from scratch in class. During the lecture time itself, we will work on practical examples in Python. As a result of the extra 1-2 hours of time spent watching the videos per week, we will aim to make the labs shorter than in other courses.
Some context: these videos are from Mike's undergraduate machine learning course, CPSC 340, which contains a lot of the same material. They were filmed in January-April 2018. You can find the accompanying slides, and some supplementary readings, here. By the end of MDS we will cover roughly everything in CPSC 340, although often skipping over the implementation details. On the other hand, we cover machine learning topics that do not appear in CPSC 340, like time series data and natural language data. In MDS we also benefit greatly from the statistical perspective of "stat stream" courses.
Video timings: video links have start times embedded in them, which is where you are supposed to start watching from. End times are specified below if you're not supposed to watch the whole video. I recommend watching the videos at 1.25x speed.
- Optimization, gradient descent
- Floating point issues, computing derivatives
- Neural networks
- Convolutional neural networks
- James, Gareth; Witten, Daniela; Hastie, Trevor; and Tibshirani, Robert. An Introduction to Statistical Learning: with Applications in R. 2014. Plus Python code and more Python code.
- Russell, Stuart, and Peter Norvig. Artificial intelligence: a modern approach. 1995.
- David Poole and Alan Mackwordth. Artificial Intelligence: foundations of computational agents. 2nd edition (2017). Free e-book.
- Kevin Murphy. Machine Learning: A Probabilistic Perspective. 2012.
- Christopher Bishop. Pattern Recognition and Machine Learning. 2007.
- Pang-Ning Tan, Michael Steinbach, Vipin Kumar. Introduction to Data Mining. 2005.
- Mining of Massive Datasets. Jure Leskovec, Anand Rajaraman, Jeffrey David Ullman. 2nd ed, 2014.
Math for ML
- Mathematics for Machine Learning
- The Matrix Calculus You Need For Deep Learning
- Introduction to Optimizers
Other ML resources
Deep learning resources
- Dive into Deep Learning, a book based on STAT 157 at UC Berkeley.
- Deep learning YouTube series by 3Blue1Brown.
- Neural Networks and Deep Learning (free online book).
- Deep Learning. Ian Goodfellow, Yoshua Bengio and Aaron Courville.
- Deep Learning with Python. Jason Brownlee.
- Stanford UFLDL tutorial (or here)
- Geoff Hinton Coursera lectures
- CS231n: Convolutional Neural Networks for Visual Recognition (Stanford)
- Grokking Deep Learning
- Practical Deep Learning For Coders, Part 1 and some more resources on their blog here
- A Guide to Deep Learning
- Awesome Deep Learning, which is a list of other resources