Image classification on fashion-MNIST
I would like to share my results (93.43% accuracy on average) on the fashion-MNIST dataset. You can find further informations about the dataset on Zalando Research and Kaggle
This dataset is a great option instead of using traditional handwritten MNIST.
Thanks to Han and Kashif!
I splitted the training data randomly in train (80%) and validation (20%). The testset of 10k images are used for final evaluation. I created 5 models with the same architecture but with random train/validation data. I only saved the weights of every model with best loss. Finally I used the models to evaluate them on the testset. The average loss/accuracy of the 5 models is the final result.
Furthermore I evaluate the generalization of the network for classifying traditional MNIST at the end.
Actually I'm focussed on a very simple architecture with less than 500,000 parameters which could run on CPU with 4GB memory. If you are interested in further results, you can find them on Zalando benchmark. If it fits in time, I will evaluate architectures e.g. DenseNet.
I'm just using two convolutional layers, batchnorm, dropout and three fully connected layers. For a detailed implementation check the model definition in the jupyter notebook.
This is the summary of the current model:
It's easy to reach an accuracy over 90% but around 92% it stucks very fast. If we take a look on the confusion matrix, we will find out that distinguishing between Shirt and T-Shirt/top is very difficult:
Here's an example where my model fails. I'm not a fashion expert. Maybe thats the reason why I don't see any difference. If you could explain why this is a shirt and not a t-shirt I would appreciate it:
Scores for training: 0.1066 loss / 95.99% accuracy.
Scores for validation: 0.1245 loss / 95.64% accuracy.
Scores for test: 0.2149 loss / 93.43% accuracy.
If you like, you can download the saved models/weights and history for fashion-MNIST.
I used the same architecture of neural network to train on traditional MNIST.
Scores for training: 0.078 loss / 99.74% accuracy.
Scores for validation: 0.161 loss / 99.61% accuracy.
Scores for test: 0.0248 loss / 99.43% accuracy.
If you like, you can download the saved models/weights and history for MNIST.
|93.43%||95.99%||95.64%||99.43%||BatchSize : 250
Epochs : 80
Data augmentation (2x)
Feel free to use this code and models for improving on your own. I would appreciate it if you give any feedback.
The easiest way to use the code examples is to download / clone the whole repository. The only thing that’s missing is the folder
data with the given subfolder structure for the input images:
data ├── test | ├── fashion_mnist | | ├── 0 | | ├── 1 | | ├── ... | | | ├── mnist | | ├── 0 | | ├── 1 | | ├── ... | ├── train | ├── fashion_mnist | | ├── 0 | | ├── 1 | | ├── ... | | | ├── mnist | | ├── 0 | | ├── 1 | | ├── ...
For getting this structure automatically I wrote a extractor finding in this jupyter notebook. This extracts the images of a MNIST IDX file.
If you have any questions or hints contact me through an issue. Thanks!