How to shuffle data for training?

[drozdvadym]

I am not sure, but all networks that was trained on two classes, converge very fast. And for the several first epochs, network recognizes only negative classes. Maybe it due to not shuffled data?

[lukeyeager]

The data is automatically shuffled for you. I was constantly forgetting to do that with CLI caffe, so I went ahead and made it the default.

Are you saying that converging quickly is a problem? I'm not sure I understand your issue.

[drozdvadym]

I created database, by parsing the directory with the next structure:

.
└── data
    └── pos
    └── neg

Where pos - images with pedestrian, and neg - without.

For training I choosed simple LeNet architecture.

And have next train log:

As you can see accuracy is very good.

But when I try to test on some samples, CNN all time classifies as neg class (see figure below).

Negative sample:

Positive sample:

[lukeyeager]

Ok so in your Accuracy/Loss graph, you can see that the validation loss is much higher than your training loss. That usually means one of two things:

1. Your dataset is not good. You might have too few images (you'll probably need at least a couple hundred examples for each category; 1000 would not be too many). Or the problem might just be too hard. Those two examples you gave seem pretty easy, but I haven't seen the rest of your dataset. I'll assume you can figure that out.
2. Or your network is not well suited for your data - you might have many parameters in your network or too few, or your random weight initialization or regularization may be off, etc. In your case, I don't think LeNet is the architecture you need to solve your problem. Your images are color and look bigger than 28x28, whereas LeNet was specifically designed to fit a dataset of small greyscale images in 10 different categories. Alexnet will probably give you a model that works much better, but then you'll have the problem of using a model that is much bigger than what you really need.

In general, those "standard networks" are not expected to solve all of your problems. They are simply a starting point to give you a few examples of the types of neural network architectures that have been used before. Once you find that neither of them solves your problem perfectly (and they usually won't), then you'll have to step into the world of designing a neural network yourself. That's the whole point of DIGITS, really - making it easier to try different model architectures and get a feel for if they're working quickly, while helping you avoid making some of the more mundane mistakes like syntax errors.

[lukeyeager]

For posterity, here is what it looks like when a model (AlexNet, in this case) fits a dataset (VOC 2007, in this case) relatively well. This dataset is really hard (widely varying image scales, huge category skew), so the accuracy doesn't go very high, but you can see that the loss on the training set and the validation set are both decreasing (so learning is happening), and they are staying close together (so the function you are learning is a good predictor for images in the validation set as well as the training set).

[drozdvadym]

Hi there!

I took LeNet architecture only as base, I've changed it for my data.

Input size is 64x128, two outputs.

Pedestrian data - 20k
Non pedestrian data - 25k.

I ask only because it look like all labels in created database are equal 0.

[drozdvadym]

I found problem:

• when you choose dataset param, first dimension is height and second is width, but example with "Mona Lisa" shows differ size (first width and second height)!

Maybe is better to show some hint for sizes?

[lukeyeager]

@drozdvadym, can you open a new issue? Thanks for reporting it - that is indeed a bug and needs to be fixed, but it doesn't belong in this thread.

[drozdvadym]

Ok