Deep Learning (for Computer Vision - CS 763) Module - Spring 2017

Course Information

Please refer to the CS763 Spring 2017 course page for general information. On this page, you will find specific information for the lectures that will be taught by Arjun Jain.

• Instructor: Arjun Jain
• Office: 216, CSE New Building
• Email: ajain@cse DOT iitb DOT ac DOT in
• Instructor Office Hours (in room 216 CSE New Building): Arjun is on campus only on Fridays (and Saturdays for the 3 weeks during this course). Meet him after class or fix an appointment over email.

Topics to be covered (tentative)

• The data-driven paradigm, feed forwards networks, back-propagation and chain rule
• CNNs and their building blocks - ReLU, MaxPool, Convolution, CrossEntropy, etc.
• Vanishing gradients, residual blocks, visualizing and understanding CNNs
• CNN applications, CNN compression (and if time permits Siamese and Triplet networks)

Learning materials and textbooks

• Lecture slides that will be regularly posted
• Deep Learning, by Ian Goodfellow and Yoshua Bengio and Aaron Courville (freely downloadable!)
• All iTorch notebooks for topics covered in class can be found here

Tutorials and other useful resources

• Learn Lua in 15 minutes
• Torch's Tensor class
• Torch's math library
• Awesome deep learning papers

Assignment

We will use Kaggle to host the leader board for the assignment. You will be able to submit your model predictions on Kaggle using your @iitb.ac.in email ids and check where you stand as compared to others in real time. Complete details about the assignment can be found here. Sample data can be downloaded from here. Due Date: 14 April 2017, 23:55.

Lecture Schedule:

Date	Topics	Slides	iTorch Notebooks
24/03/2017	Brief history/achievements of DL based algorithms The data driven paradigm Classification using k-nearest neighbor algorithm Classification using a linear classifier Optimization using vanilla gradient descent Feed forward networks	Slides	Vanilla kNNs Numerical Gradients
25/03/2017	Chain rule/backward propagation Torch7 nn.Module Activation functions	Slides	ReLU Activation Funciton
31/03/2017	Fully Conncted Layer Convolution Layer Fully Connected as Convolution	Slides	Fully Conncted Layer Convolution Layer
01/04/2017	Max Pool Layer Cross Entropy Layer Dropout Layer Data Preprocessing and Augmentation Weight Initialization Baby Sitting the Learning Process Hyper-parameter Optimization	Slides	Max Pool Layer Dropout Layer Weight Initialization
07/04/2017	CNN Case Studies Myth buster: You need a lot of data to train ConvNets (false!) Visualizing and understanding ConvNets	Slides
08/04/2017	Visualizing and understanding ConvNets (cont.) Art/Fun with NNs (NeuralArt, DeepDream) CNN Compression	Slides