STAT 991: Topics in deep learning (UPenn)
STAT 991: Topics in Deep Learning is a seminar class at UPenn started in 2018. It surveys advanced topics in deep learning based on student presentations.
Potential topics: Uncertainty quantification, Adversarial Examples, Symmetry, Theory and Empirics, Interpretation, Fairness, ...
See section 6.1 of my lecture notes for a collection of materials.
Theory and empirical wonders
Climate, energy, healthcare...
Course on Coursera. A good way to learn the basics.
Convenient ways to run code online: https://colab.research.google.com/notebooks/welcome.ipynb, https://www.kaggle.com/kernels
Keras is a user-friendly language for DL. Interfaces to R, see this book
Materials from previous editions
The materials draw inspiration from many sources, including David Donoho's course Stat 385 at Stanford, Andrew Ng's Deep Learning course on deeplearning.ai, CS231n at Stanford, David Silver's RL course, Tony Cai's reading group at Wharton. They may contain factual and typographical errors. Thanks to several people who have provided parts of the notes, including Zongyu Dai, Georgios Kissas, Jane Lee, Barry Plunkett, Matteo Sordello, Yibo Yang, Bo Zhang, Yi Zhang, Carolina Zheng. The images included are subject to copyright by their rightful owners, and are included here for educational purposes.
- Lecture notes. (~170 pages, file size ~30 MB.)
Compared to other sources, these lecture notes are aimed at people with a basic knowledge of probability, statistics, and machine learning. They start with basic concepts from deep learning, and aim to cover selected important topics up to the cutting edge of research.
The entire latex source is included, encouraging reuse (subject to appropriate licenses).
Topics: sequential decision-making (from bandits to deep reinforcement learning), distributed learning, AutoML, Visual Question Answering.
Lecture 1: Bandits. Presented by Edgar Dobriban.
Lecture 2: Contextual Bandits. Presented by Bo Zhang.
Lecture 2b: Contextual Bandits for Mobile Health. Presented by Halley Young.
Lectures 4-9: Reinforcement learning following David Silver's course.
Lecture 11: Hierarchical Reinforcement learning. Presented by Barry Plunkett.
Lecture 12: AutoML. Presented by Yi Zhang.
Lecture 13: Visual Question Answering. Presented by Reno Kriz.
Lecture 13b: Visual Question Answering: Part 2. Presented by Soham Parikh.
Topics: basics (deep feedforward networks, training, CNNs, RNNs). Generative Adversarial Networks, Learning Theory, Sequence Learning, Neuroscience, etc.
Lectures 1-3: Lectures based on Edgar Dobriban's notes.
Lecture 4: Generative Adversarial Networks. Presented by Zilu Zhou.
Lecture 5: Theory for Generative Adversarial Networks. Presented by Hadi Elzayn.
Lecture 6: Learning Theory for Neural Networks. Presented by Jacob Seidman.
Lecture 7: Gradient Based Optimization. Presented by Matteo Sordello.
Lecture 8: Sequence Learning. Presented by Carolina Zheng.
Lecture 9: Robotics. Presented by Ty Nguyen.
Lecture 10: Autoencoders, Physics-Informed Neural Networks. Presented by Yibo Yang and Georgios Kissas.
Lecture 11: Neuroscience Inspired Deep Learning. Presented by Huy Le.
Lecture 12: Approximation and Estimation for Deep Learning Networks. Guest lecture by Jason Klusowski.
Lecture 13: Deep Learning in Marketing Research. Presented by Mingyung Kim.