Part 1: Artificial Neural Networks (ANN)

Datasets & Templates:

Artificial-Neural-Networks

Additional Reading:

• Yann LeCun et al., 1998, Efficient BackProp
• By Xavier Glorot et al., 2011 Deep sparse rectifier neural networks
• CrossValidated, 2015, A list of cost functions used in neural networks, alongside applications
• Andrew Trask, 2015, A Neural Network in 13 lines of Python (Part 2 – Gradient Descent)
• Michael Nielsen, 2015, Neural Networks and Deep Learning

Part 2: Convolutional Neural Networks (CNN)

Datasets & Templates:

Convolutional-Neural-Networks

Additional Reading:

• Yann LeCun et al., 1998, Gradient-Based Learning Applied to Document Recognition
• Jianxin Wu, 2017, Introduction to Convolutional Neural Networks
• C.-C. Jay Kuo, 2016, Understanding Convolutional Neural Networks with A Mathematical Model
• Kaiming He et al., 2015, Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
• Dominik Scherer et al., 2010, Evaluation of Pooling Operations in Convolutional Architectures for Object Recognition
• Adit Deshpande, 2016, The 9 Deep Learning Papers You Need To Know About (Understanding CNNs Part 3)
• Rob DiPietro, 2016, A Friendly Introduction to Cross-Entropy Loss
• Peter Roelants, 2016, How to implement a neural network Intermezzo 2

Part 3: Recurrent Neural Networks (RNN)

Datasets & Templates:

Recurrent-Neural-Networks Homework-Challenge

Additional Reading:

• Oscar Sharp & Benjamin, 2016, Sunspring
• Sepp (Josef) Hochreiter, 1991, Untersuchungen zu dynamischen neuronalen Netzen
• Yoshua Bengio, 1994, Learning Long-Term Dependencies with Gradient Descent is Difficult
• Razvan Pascanu, 2013, On the difficulty of training recurrent neural networks
• Sepp Hochreiter & Jurgen Schmidhuber, 1997, Long Short-Term Memory
• Christopher Olah, 2015, Understanding LSTM Networks
• Shi Yan, 2016, Understanding LSTM and its diagrams
• Andrej Karpathy, 2015, The Unreasonable Effectiveness of Recurrent Neural Networks
• Andrej Karpathy, 2015, Visualizing and Understanding Recurrent Networks
• Klaus Greff, 2015, LSTM: A Search Space Odyssey
• Xavier Glorot, 2011, Deep sparse rectifier neural networks

Part 4: Self Organizing Maps (SOM)

Datasets & Templates:

Self-Organizing-Maps Mega-Case-Study

Additional Reading:

• Tuevo Kohonen, 1990, The Self-Organizing Map
• Mat Buckland, 2004?, Kohonen's Self Organizing Feature Maps
• Nadieh Bremer, 2003, SOM – Creating hexagonal heatmaps with D3.js

Part 5: Boltzmann Machines (BM)

Datasets & Templates:

Boltzmann-Machines

Additional Reading:

• Yann LeCun, 2006, A Tutorial on Energy-Based Learning
• Jaco Van Dormael, 2009, Mr. Nobody
• Geoffrey Hinton, 2006, A fast learning algorithm for deep belief nets
• Oliver Woodford, 2012?, Notes on Contrastive Divergence
• Yoshua Bengio, 2006, Greedy Layer-Wise Training of Deep Networks
• Geoffrey Hinton, 1995, The wake-sleep algorithm for unsupervised neural networks
• Ruslan Salakhutdinov, 2009?, Deep Boltzmann Machines

Part 6: AutoEncoders (AE)

Datasets & Templates:

AutoEncoders

Additional Reading:

• Malte Skarupke, 2016, Neural Networks Are Impressively Good At Compression
• Francois Chollet, 2016, Building Autoencoders in Keras
• Chris McCormick, 2014, Deep Learning Tutorial - Sparse Autoencoder
• Eric Wilkinson, 2014, Deep Learning: Sparse Autoencoders
• Alireza Makhzani, 2014, k-Sparse Autoencoders
• Pascal Vincent, 2008, Extracting and Composing Robust Features with Denoising Autoencoders
• Salah Rifai, 2011, Contractive Auto-Encoders: Explicit Invariance During Feature Extraction
• Pascal Vincent, 2010, Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion
• Geoffrey Hinton, 2006, Reducing the Dimensionality of Data with Neural Networks