Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CUDA kernel error (unspecified launch failure) #40

Closed
ghost opened this issue Jan 12, 2015 · 4 comments
Closed

CUDA kernel error (unspecified launch failure) #40

ghost opened this issue Jan 12, 2015 · 4 comments

Comments

@ghost
Copy link

ghost commented Jan 12, 2015

Hi, I hit an error when I want to pass my image through net on GPU (Tesla k20c / Ubuntu 1204/Matlab 2014a).
here's the error

im2col: CUDA kernel error (unspecified launch failure)
Error using gpuArray/max
An unexpected error occurred during CUDA execution. The CUDA error was:
unspecified launch failure

Error in vl_nnrelu (line 17)
  y = max(x, single(0)) ;

Error in vl_simplenn (line 165)
      res(i+1).x = vl_nnrelu(res(i).x) ;
...

Both net and input image were moved to gpu. Thanks for help!
@peiyunh
Copy link

peiyunh commented Jan 14, 2015

You may want to try attaching cuda-gdb to Matlab and see where the problem really is.

lenck added a commit that referenced this issue Feb 10, 2015
@lenck
Fixed issue #40 : error with invalid data after switching GPU.
d39e0d2 
When gpuDevice called in Matlab, the GPU is re-initialised
and all data become invalid [1]. Therefore, the fix checks
whether the nnconv internal buffers are valid and in case
they are not, it re-initialises them and the cublas handle.

[1] http://uk.mathworks.com/help/distcomp/mxgpuisvalidgpudata.html
@lenck
Copy link
Contributor

lenck commented Feb 10, 2015

Hi @guanggz, can you check if this solves the issue for you?
I've got similar errors, but only when I've swtiched the GPUs. So this commit should fix that, but I'm not sure if it is your case....

@ghost
Copy link
Author

ghost commented Feb 11, 2015

@lenck I would guess so. I will try to reconstruction the problem and come back to it once I have access to that machine.

@vedaldi
Copy link
Contributor

vedaldi commented Feb 19, 2015

beta9 should be much more resilient to switching GPUs. I think this problem is likely fixed.

@vedaldi vedaldi closed this as completed Feb 19, 2015
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@vedaldi @lenck @peiyunh