ResNet in tensorflow for CIFAR-100, CIFAR-10
Branch: master
Clone or download
Latest commit 904c6c6 Sep 28, 2017
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.ipynb_checkpoints submitted May 5, 2016
report submitted May 5, 2016
resource submitted May 5, 2016
summary submitted May 5, 2016
.gitattributes Create .gitattributes May 10, 2016
.gitignore start Apr 25, 2016
README.md Update README.md Sep 27, 2017
__init__.py tensorflow tutorial Apr 25, 2016
autogit.bash test-autogit Apr 25, 2016
batch_norm.py accuracy 60 May 1, 2016
checkpoint submitted May 5, 2016
cifar10.py accuracy 60 May 1, 2016
cifar10_eval.py try Apr 30, 2016
cifar10_input.py try Apr 30, 2016
cifar10_input_test.py tensorflow tutorial Apr 25, 2016
cifar10_train.py accuracy 60 May 1, 2016
convnet.py Update convnet.py May 11, 2016
data_utils.py first Apr 25, 2016
dataset_utils.py accuracy 60 May 1, 2016
kcnn.py submitted May 5, 2016
kmeans.py submitted May 5, 2016
mp2.pdf first Apr 25, 2016
redo.py submitted May 5, 2016
resnet.py try Apr 30, 2016
summary.py accuracy 60 May 1, 2016
test.ipynb submitted May 5, 2016
tryhere.ipynb submitted May 5, 2016

README.md

Fast ResNet for CIFAR

This ResNet spends 3 hours to train on a modern CPU. It can reach 63% on CIFAR100 coarse 20 classes task.
This residual neural network is different from the original paper in there ways:

  • Only have 13 layers, which the original paper didn't studied.
  • No ReLU before subsampling convolutional layer, which improve accuracy by 3%
  • BatchNorm is done before addition, which improve accuracy a little bit.

This ResNet-13 can't beat ResNet-18/34/150/1000 layers residual nets in the long run, however, more efficient with non-sufficient training time, interestingly.
Details are shown here. Archtecture shown at the bottom.

results on CIFAR-10/CIFAR-100

Traning 3 hours on CPU:

Acc. CIFAR-10 CIFAR-100
Alexnet 82% -
Mimic Learning - 50%
2-layer NN with PCA and Kmeans 78% 56%
ResNet-13 (this repo) 84% 63%

How to run

python redo.py /path/to/CIFAR100python/

Features

  • Output layers contain 20 labels
  • Using tensorflow
  • number of filters
  • iterations
  • learning rate
  • batch size
  • regularization strength
  • number of layers
  • optimization methods
  • drop out
  • initialization
  • hidden neurons
  • filter size of convolution layers
  • filter size of pooling layers

architecture

arch