cnn-fashion-mnist

A repository contains various CNN architecture Deep Learning experiment for Fashion-MNIST Data

Introduction

Fashion-MNIST is a dataset of Zalando's article images—consisting of a training set of 60,000 examples and a test set of 10,000 examples. Each example is a 28x28 grayscale image, associated with a label from 10 classes

Fashion MNIST Data

Data Loading

We separate the data become Training set and Testing set based on source

Fashion-MNIST Dataset

For training purpose, the dataset is augmented with several transform

• Resize Image resize from 32x32 become 96x96
• RandomHorizontalFlip Random flip image horizontally
• RandomRotation Random rotation image with max 10 degree
• GrayScale(3) Make image from grayscale 1 channel to 3 channel

Training

Initialization

Hyperparameter Tuning

• Learning Rate
• Batch Size
• Number of Epoch

Implementation

• Criterion

criterion = nn.CrossEntropyLoss()

• Optimizer

optimizer = optim.SGD(net.parameters(), lr=0.01, momentum=0.9)

• Scheduler

scheduler = optim.lr_scheduler.StepLR(optimizer, step_size=10, gamma=0.2)

Testing Result

CNN Architecture	Model size (MB)	Best Accuracy (%)
VGG-16	1030	93.25
ResNet-18	89.25	94.17
ResNet-50	30	93.11
ResNet-152	445	95.39
DenseNet-121	54	95.52
DenseNet-161	203.24	95.42

Optimize Result

Test Time Augmentation

Augment the data for testing process. There are two types data, first is original and second is horizontal flip. Inference based averaging two output from augment test data

• Original accuracy = 95.52 %
• After augmented = 95.59 %

Ensemble Method

For Ensemble the model I use Bagging method, it means we only process all output from several models

Here the result

• Ensemble ResNet-152 + DenseNet-121

  • w₁ = 0.5, w₂ = 0.5

    Testing accuracy result is 95.75%

  • w₁ = 0.6, w₂ = 0.4

    Testing accuracy result is Accuracy 95.80 %

• Ensemble DenseNet-121 + DenseNet-161

  • w₁ = 0.5, w₂ = 0.5

    Testing accuracy result is 95.95%

  • w₁ = 0.6, w₂ = 0.4

    Testing accuracy result is Accuracy 96.01 %

  • w₁ = 0.4, w₂ = 0.6

    Testing accuracy result is Accuracy 95.86 %

  • w₁ = 0.7, w₂ = 0.3

    Testing accuracy result is Accuracy 95.95 %

• Ensemble DenseNet-121 + DenseNet-161 + ResNet-152

  • w₁ = 0.33, w₂ = 0.33, w₃ = 0.33 (mean)

    Testing accuracy result is 95.71%

  • w₁ = 0.4, w₂ = 0.3, w₃ = 0.3

    Testing accuracy result is 95.71%

Conclusion

• CNN Architecture component consideration

  • Parameter size
  • Capacity

• Based several training experiment, here the technique can increase accuracy

  • Data Augmentation (Resizing, RandomHorizontalFlip, RandomRotation)
  • Ensemble Method

• Transfer learning speed up training process and increase testing accuracy