Implementations of CNNs, RNNs and cool new techniques in deep learning from scratch
Switch branches/tags
Nothing to show
Clone or download
parasdahal Merge pull request #7 from echatzidaki/master
Improved code consistency and fixed a minor bug.
Latest commit 5c9ae02 Sep 10, 2018

README.md

deepnet

Implementations of CNNs, RNNs and cool new techniques in deep learning

Note: deepnet is a work in progress and things will be added gradually. It is not intended for production, use it to learn and study implementations of latest and greatest in deep learning.

What does it have?

Network Architecture

  1. Convolutional net
  2. Feed forward net
  3. Recurrent net (LSTM/GRU coming soon)

Optimization Algorithms

  1. SGD
  2. SGD with momentum
  3. Nesterov Accelerated Gradient
  4. Adagrad
  5. RMSprop
  6. Adam

Regularization

  1. Dropout
  2. L1 and L2 Regularization

Cool Techniques

  1. BatchNorm
  2. Xavier Weight Initialization

Nonlinearities

  1. ReLU
  2. Sigmoid
  3. tanh

Usage

  1. virtualenv .env ; create a virtual environment
  2. source .env/bin/activate ; activate the virtual environment
  3. pip install -r requirements.txt ; Install dependencies
  4. python run_cnn.py {mnist|cifar10} ; mnist for shallow cnn and cifar10 for deep cnn