Running net2 without a GPU

[marsplus]

First I really appreciate your effort in making this tutorial and nolearn wrappers. With nolearn I can just twist deep neural networks as easily as with scikit-learn.

When I am going through your tutorial I encounter a problem. This problem happens when I fit X, y in net2, namely net2.fit(X, y). I got an error message, "CudaNdarrayType only supports dtype float32 for now. Tried using dtype float64 for variable None". I don't have GPU on my Mac so I just modify the code from:

Conv2DLayer = layers.cuda_convnet.Conv2DCCLayer
MaxPool2DLayer = layers.cuda_convnet.MaxPool2DCCLayer

into:

Conv2DLayer = layers.Conv2DLayer
MaxPool2DLayer = layers.MaxPool2DLayer

Unfortunately new error occurs, and the error message is, "the batch size in the image (48) at run time is different than at build time (128) for the ConvOp."

I have already set the batch size to 128, but there is some weird batch size "48" in the error message.

Could you please give me some advice about this problem? Should I give up running the code on my Mac and find a GPU?

[BenjaminBossan]

Update2: Okay, the issue is that train_test_split will split the incoming data. After that split, the number of samples still needs to be a multiple of the batch size. For regression, the KFold cross validator from sklearn is used. By setting eval_size of the neural net to the appropriate size, one can fix this problem. However, for classification, StratifiedKFold is used, which makes the size of the train/test split unpredictable. For that, I see no easy fix right now.

Update1: The issue seems to be more complicated now. The remainder depends on the train_test_split and therefore behaves somewhat unpredictably. Still investigating...

I had a similar problem once, maybe the same solution applies. Back then, the layer expected the batch sizes to always have the same size. If the iterator runs out of samples, it will yield the remaining number of samples (in your case that would be 48 remaining -- change the batch size and check if this number changes accordignly).
A quick solution would be

• to set the batch size to a number such that there is no remainder
• to change the batch iterator so that it does not yield the remaining samples

Hope this helps. A fix in Lasagne/nolearn would be much appreciated, Daniel :)

[marsplus]

@BenjaminBossan Thanks for your reply. Since in this tutorial the number of training data is 2140, I modified eval_size to 0.1, and then changed input_shape to input_shape=(214, 1, 96, 96). In addition, I also modified batch_iterator=BatchIterator(batch_size=128) to batch_iterator=BatchIterator(batch_size=214) in nolearn/lasagne.py. In the end, the net2 could start training.

[dnouri]

Anyone feel like making a pull request for nolearn.lasagne that adds an option to BatchIterator so that it skips remainder batches that are smaller than batch_size?

[dnouri]

There's now a flag called force_even in BatchIterator, which deals with this problem in a much more convenient way. See dnouri/nolearn#11

[dnouri]

Forget about forced_even. It's totally unnecessary. In this example, just set

    # ...
    input_shape=(None, 1, 96, 96),
    # ...

See dnouri/nolearn#8 (comment)