cuDNN v2 out-of-memory CUDNN_STATUS_SUCCESS (8 vs. 0) CUDNN_STATUS_EXECUTION_FAILED

[xiangyangli-cn]

sorry to trouble!
I just downdload the latest caffe-master and cudnn v2,but this occour:
cudnn_conv_layer.cu:65] Check failed: status == CUDNN_STATUS_SUCCESS (8 vs. 0) CUDNN_STATUS_EXECUTION_FAILED

How can i fix this problem
Thanks!

[dshayden]

I have the same problem, can be reproduced when running either of the cifar10 examples (train_quick.sh or train_full.sh), but the mnist example works fine on GPU, as does NVIDIA's cudnn-sample-v2 code.

[shelhamer]

Sorry about that -- there should be a fix shortly for the automatic selection. This is happening because cuDNN is selecting the fastest but memory consuming algorithm so that the GPU runs out of memory. We're also figuring out the right way to expose the manual choice of convolution algorithm in the model definition.