Stat 415 Spring 2023

Stat 415 course materials

Course Lectures

Lecture notes can be found on the course canvas website.

Lecture	Date	Material	Readings
Week 1, Tuesday	March 28	No class. Watch Lecture 0 about installing python.	Scikitlearn Scikitlearn 2 Pandas Matplotlib
Week 1, Wednesday	March 29	Data Clearning, EDA, Class Imbalance, Imputation	EDA Data Imbalance All models are wrong What's wrong with social sciences Future of Data Analysis
Week 2, Monday	April 3	Information Theory Principle Componenet Analysis (PCA)	Information Theory (2.1-2.5) PCA code PCA Theory Information Theory & Physics
Week 2. Wednesday	April 5	Clustering, K-Means, K-Means++, DBSCAN, Expectation Maximization (EM)	K-means overview and code K-means clustering note K-means & EM Code K-means theory K-means and optimization theory DB Scan Wikipedia
Week 3, Monday	April 10	Recommender Systems, Popularity Filtering, Content-based Filtering, Collaborative Filtering	Stanford Slides Textbook Chapter Starter Code for Collab Filtering Starter for content filtering More content based filtering More collaborative filtering Matrix Factorization Textbook
Week 3, Wednesday	April 12	Recommender Systems Part 2
Week 4, Monday	April 17	Linear Regression, Gradient Descent	Toronto slides Boyd 9.3 (p. 466) Gradient Descent SGD convergence rate Andrew Ng's notes (p. 1-15) Linear Regression code
Week 4, Wednesday	April 19	Classification, L1 & L2 Regularization, Cross Validation, F1-Score, Precision, AIC, BIC	Classification in PyTorch Andrew Ng's notes (p. 16-21) More Andrew Ng notes Regularization notes Lasso code Regularization Code Model metrics L1 vs L2 theory
Week 5, Monday	April 24	Random Forests, Decision Trees	Decision Trees Advanced Decision Trees Random Forests Textbook Random Forests Code Random Forests simple code Causal Forests Tuning Forests
Week 5, Wednsday	April 26	Deep Learning. Activation, Initialization, Advanced Optimizer	Textbook Chapter Deep Learning Code Andrej Karpathy YouTube Lecture Backprop MLP Notes Dropout Xavier Derivation More coding
Week 6, Monday	May 1	Convolutional Neural Nets. Batch Norm. Max Pooling	Code More code Convolution Tutorial Conv Nets Long tutorial Andrej Karpathy YouTube Lecture ImageNet Paper Stanford Notes Conv net for the 2020s Res Nets
Week 6, Wednesday	May 3	Recurrent Neural Nets, Padding Cross Validation on Time Series Autoregressive models, Machine Translaiton	Alex Graves Thesis Seq2Seq paper CharRNN RNN coded from scratch RNN code GRU Nets Paper Explainability and RNNs Paper Vanishing Gradients Textbook
Week 7, Monday	May 8	Transfer Learning. Fine Tuning. MAML. Prototypical Nets	Prototypical Nets MAML MAML doesn't work Fine tuning code Clustering With Bregman
Week 7, Wednsday	May 10	Black Box Attacks. Interpretability. Integrated Gradients	Integrated Gradients SmoothGrad LIME PDP and ALE Shapley Values Intriguing properties of neural networks Saliency Maps
Week 8, Monday	May 15	Unsupervised Learning. Variational Autoencoders. Maximum Liklihood	VAEs explained From VAE to VQVAE VAE Code VAE theory Posterior Collapse
Week 8, Wednesday	May 17	Generative Adversarial Nets. Diffusion Models. SD Edit	GANs YouTube - inventor of GANs presentation W-GAN Style GAN Simple GAN code More GAN code Diffusion Models Diffusion Code
Week 9, Monday	May 22	Everything else you need to know about. SVM, Kernel Methods, Boosting More on time series Pruning. Distillation. Quantization. Bias-Variance tradeoff Double Descent Statistics on how to live a good life.	Kernel Regression Gradient Boosting theory Gradient Boosting SVM slides Pruning Distillation & Quantization Dobule Descent Double Descent interactive explanation Time Series Code Time Series
Week 9, Wednesday	May 24	Towards AGI Transformers, RL, How Chat GPT works. AI scaling laws	Deep RL from human prefrences Introduction to RL More RL GPT Illustrated GPT Transformers Annotated Transformer Scaling Laws
Week 10, Monday	May 29	Holiday, no class	Memorial Day
Week 10, Wednesday	May 31	Final exam. You may start exam at any time on May 31 and will have three hours to complete it from the time you choose to start. No class.	Best foods for studying You and Your research Writing papers Data sceience interviews Shannon on creativity Best places to visit in the US

Homeworks and Due Dates

Project title	Date released	Due date
Fraud Detection	March 29	April 14 (2 weeks)
Reccommender Systems	April 14	May 2 (2.5 weeks)
Drunk Driving Demographics	May 2	May 12 (1.5 weeks)
Explainability & Animal Images	May 12	May 26 (2 weeks)
Final Exam (Coding)	May 31	May 31 (3 hours)