how to prepare CIFAR 100 and CIFAR 10 training and testing data?

[qianguih]

Hi, there,
I am trying to re-run provided CIFAR 10 and CIFAR 100 examples. When I use the provided CIFAR 10 training and testing data, it can produce the same results as introduced. However, when I feed programs with the CIFAR 10 and CIFAR 100 data I manually converted, the accuracies decreased a lot.
Specifically, first, I downloaded the matlab version CIFAR 10 and CIFAR 100 data and wrote them into jpg images. Then, I converted these images into RecordIO format data following the same process provided here https://github.com/dmlc/mxnet/blob/master/doc/python/io.md.
Did I miss something? Is there any other pre-processing operations needed? How did you guys prepare these data?
Do anyone have some suggestions? Thank you in advance.

[tqchen]

Yes, this procedure should be the same. Please make sure you shuffle the data before pack them

[qianguih]

Hi, Tianqi,
Thanks for your reply.
https://github.com/dmlc/mxnet/blob/master/doc/python/io.md tells that the images would be shuffled automatically. Attached is my image list files. Is that ok?
Is it possible that I should change the parameter when calling im2rec? Because when I packing the images from image list, it reported the JPEG encoding quality is 80. Should I specify it to be 100? Would this matter?

cifar_10_test.txt
cifar_10_train.txt
cifar_100_train.txt
cifar_100_test.txt

[tqchen]

You want to shuffle the list before you feed them into the record packer

[tqchen]

The shuffle of recordReader is only done in a local manner, which means it only shuffles data within each block

[qianguih]

Sorry, I am a little confused about "local" and "each block". Could you be more specific? Does this mean I still need to make numbers in the first column in train.lst randomly organized?

[tqchen]

Yes, I mean you to shuffle the list, before you pack, so the same labels won't occur consecutively

[qianguih]

Hi, I tried to shuffle the first column in the train.lst. However, the accuracy seems to stick around 83% after 20 epochs. Attached is my train.lst file for CIFAR 10. Is there anything wrong with it?

train.txt

[qianguih]

By the way, I shuffled .lst files using the bash command shuf. Is there any better way to do this?

[tqchen]

I do not mean shuffle first column, but shuffle the lines in your train.lst before you pack

[qianguih]

Sorry for the confusion. The lines in train.lst are shuffled.
And I also set the quality parameter to be 100 during packing. But the accuracy is only 84% for CIFAR 10 after 30 epochs.

[qianguih]

Btw,is there any API provided to extract the activations in a specific layer in a given model?

[tqchen]

yes, c.f. https://github.com/dmlc/mxnet/blob/master/example/imagenet/predict-with-pretrained-model.ipynb

[qianguih]

Thank you! You guys are awesome.

[max0x]

I encountered a similar problem. I converted MNIST binary data to jpg images, and then packed jpg images to an ImageRecord binary file. However, the training accuracy was always about 0.114844, which is just a random guess.

I referred to http://mxnet.io/packages/python/io.html#create-dataset-using-recordio to pack images to ImageRecord binary. My command is:
python ~/packages/mxnet/tools/im2rec.py --color 0 --quality 100 --shuffle True "" jpg

Then I used a MLP for training:

import mxnet as mx
import logging
import os

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

home = os.path.expanduser("~")
prefix = home + "/dataset/"

train = mx.io.ImageRecordIter(
    path_imgrec=prefix + "mnist/train-images-idx3.rec",
    data_shape=(1,28,28),
    batch_size = 128,
    shuffle=True,
)

data = mx.symbol.Variable("data")
fc1  = mx.symbol.FullyConnected(data=data, num_hidden=128)
act1 = mx.symbol.Activation(data=fc1, act_type="relu")
fc2  = mx.symbol.FullyConnected(data=act1, num_hidden=64)
act2 = mx.symbol.Activation(data=fc2, act_type="relu")
fc3  = mx.symbol.FullyConnected(data=act2, num_hidden=10)
mlp  = mx.symbol.SoftmaxOutput(data=fc3, name="softmax")

model = mx.model.FeedForward(
    symbol = mlp,
    num_epoch = 20,
    learning_rate = .1,
)

model.fit(X=train,
          eval_metric=["accuracy"],
          batch_end_callback=mx.callback.Speedometer(128),
          logger=logger)

The logging info is always like this:
INFO:__main__:Start training with [cpu(0)] INFO:root:Epoch[0] Batch [50] Speed: 53125.66 samples/sec Train-accuracy=0.113125 INFO:root:Epoch[0] Batch [100] Speed: 53541.67 samples/sec Train-accuracy=0.111406 INFO:root:Epoch[0] Batch [150] Speed: 21821.71 samples/sec Train-accuracy=0.118438 INFO:root:Epoch[0] Batch [200] Speed: 52519.27 samples/sec Train-accuracy=0.106875 INFO:root:Epoch[0] Batch [250] Speed: 55297.81 samples/sec Train-accuracy=0.116562 INFO:root:Epoch[0] Batch [300] Speed: 21163.96 samples/sec Train-accuracy=0.110156 INFO:root:Epoch[0] Batch [350] Speed: 53972.50 samples/sec Train-accuracy=0.115625 INFO:root:Epoch[0] Batch [400] Speed: 41040.34 samples/sec Train-accuracy=0.110937 INFO:root:Epoch[0] Batch [450] Speed: 64686.98 samples/sec Train-accuracy=0.105625 INFO:__main__:Epoch[0] Resetting Data Iterator INFO:__main__:Epoch[0] Time cost=2.262 INFO:root:Epoch[1] Batch [50] Speed: 53814.96 samples/sec Train-accuracy=0.111719 INFO:root:Epoch[1] Batch [100] Speed: 53497.82 samples/sec Train-accuracy=0.112656 INFO:root:Epoch[1] Batch [150] Speed: 20033.24 samples/sec Train-accuracy=0.115781 INFO:root:Epoch[1] Batch [200] Speed: 52946.37 samples/sec Train-accuracy=0.115625 INFO:root:Epoch[1] Batch [250] Speed: 51084.73 samples/sec Train-accuracy=0.113437 INFO:root:Epoch[1] Batch [300] Speed: 21558.19 samples/sec Train-accuracy=0.107344 INFO:root:Epoch[1] Batch [350] Speed: 52814.80 samples/sec Train-accuracy=0.112969 INFO:root:Epoch[1] Batch [400] Speed: 38230.61 samples/sec Train-accuracy=0.115312 INFO:root:Epoch[1] Batch [450] Speed: 66028.31 samples/sec Train-accuracy=0.108281

Any suggestions? Thanks!

train-images-idx3.lst.txt