Deep Learning from Scratch

This project is an implementation of a neural network only with numpy, without using other deep learning frameworks such as tensorflow or pytorch.

The book Deep Learning from Scratch was used for reference.

Contents

Chapter 1 was introduction to Python, and thus was omitted.

Chapter 2 through 7 is about implementing CNNs.

Chapter 2. Perceptron

Chapter 3. Neural Networks

• Activation Functions

Chapter 4. Training Neural Networks

• Loss Functions
• Numerical Differentiation
• Gradient Descent

Chapter 5. Backpropagation

• Computational Graphs
• Chain Rule and Backpropagation

Chapter 6. Training Techniques

• Optimizers
• Initializing Weights
• Batch Normalization
• Dropout
• Hyperparameter Tuning

Chapter 7. Convolutional Neural Networks

• Conv/Pooling Layer
• Visualizing CNN

Chapter 8. Deep Learning (Image Recognition)

• Deeper Networks
• History of Image Recognition
• Speeding Up Training
• Applications of Deep Learning