A convolutional nerual net trained from scratch in pure python/numpy on the CIFAR 10 image dataset. I was able to achieve 70% accuracy on both validation and test sets trained with a shallow architecture on my 2013 Macbook's CPU (yikes).
I used Tensorboard without Tensorflow for graphing accuracy and cost during training. This information is written to the out.graph file.
Clone the repository and run python main.py after installing dependencies. The CIFAR 10 dataset should download into the correct folder automatically. There are some boolean variables at the top of the main.py which can be set to switch between training and testing.
Here is a list of some features I implemented for faster training:
- Adam optimization algorithm
- Batchnorm after every convolution
- Xavier weight initialization (essential)
| Layer | Details |
|---|---|
| Conv | Stride: 1 Pad: 1 Weight: 3x3x16 |
| Batchnorm | |
| Relu | |
| Pool | Stride:2 Width/Height: 2 |
| Conv | Stride: 1 Pad: 1 Weight: 3x3x16 |
| Batchnorm | |
| Relu | |
| Pool | Stride:2 Width/Height: 2 |
| Conv | Stride: 1 Pad: 1 Weight: 3x3x16 |
| Batchnorm | |
| Relu | |
| Pool | Stride:2 Width/Height: 2 |
| Fully Connected | Reshape to column vector |
| Softmax |
