Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failing GPU tests on CUDA 8 #40

Closed
mattndu opened this issue Jun 18, 2016 · 3 comments
Closed

Failing GPU tests on CUDA 8 #40

mattndu opened this issue Jun 18, 2016 · 3 comments

Comments

@mattndu
Copy link

mattndu commented Jun 18, 2016

I'm running on a GTX 1070. This is compiled with CUDA 8.0 release candidate.

The ./test_gpu script fails with the following error:

Running GPU tests
terminate called after throwing an instance of 'std::runtime_error'
  what():  Error: compute_ctc_loss in small_test, stat = execution failed
Aborted (core dumped)

Attaching a debugger, I see:

(gdb) run
Starting program: /warp-ctc/build/test_gpu 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Running GPU tests
[New Thread 0x7fffef84b700 (LWP 10325)]
[New Thread 0x7fffef04a700 (LWP 10326)]
terminate called after throwing an instance of 'std::runtime_error'
  what():  Error: compute_ctc_loss in small_test, stat = execution failed

Program received signal SIGABRT, Aborted.
0x00007ffff6d55c37 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56  ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  0x00007ffff6d55c37 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x00007ffff6d59028 in __GI_abort () at abort.c:89
#2  0x00007ffff7660535 in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3  0x00007ffff765e6d6 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x00007ffff765e703 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x00007ffff765e922 in __cxa_throw () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x0000000000403e33 in throw_on_error (message=0x4097f8 "Error: compute_ctc_loss in small_test", status=<optimized out>) at /storage/deep_learning/warp-ctc/tests/test.h:11
#7  small_test () at /storage/deep_learning/warp-ctc/tests/test_gpu.cu:63
#8  0x000000000040360f in main () at /storage/deep_learning/warp-ctc/tests/test_gpu.cu:333
@ekelsen
Copy link
Contributor

ekelsen commented Jul 1, 2016

The default compilation flags don't compile for the Pascal architecture, you'll need to modify CMakelists.txt to include sm_60.

@mattndu
Copy link
Author

mattndu commented Jul 4, 2016

That worked, thank you! For reference, here is what I added to CMakeLists.txt:
set(CUDA_NVCC_FLAGS "${CUDA_NVCC_FLAGS} -gencode arch=compute_60,code=sm_60")

@ahbon123
Copy link

ahbon123 commented Jul 17, 2019

I installed cuda10.1, RTX 2080 GPU, but still have this problem.

# need to be at least 30 or __shfl_down in reduce wont compile
set(CUDA_NVCC_FLAGS "${CUDA_NVCC_FLAGS} -gencode arch=compute_30,code=sm_30 -O2")
set(CUDA_NVCC_FLAGS "${CUDA_NVCC_FLAGS} -gencode arch=compute_35,code=sm_35")

set(CUDA_NVCC_FLAGS "${CUDA_NVCC_FLAGS} -gencode arch=compute_50,code=sm_50")
set(CUDA_NVCC_FLAGS "${CUDA_NVCC_FLAGS} -gencode arch=compute_52,code=sm_52")

IF (CUDA_VERSION GREATER 7.6)
    set(CUDA_NVCC_FLAGS "${CUDA_NVCC_FLAGS} -gencode arch=compute_60,code=sm_60")
    set(CUDA_NVCC_FLAGS "${CUDA_NVCC_FLAGS} -gencode arch=compute_61,code=sm_61")
    set(CUDA_NVCC_FLAGS "${CUDA_NVCC_FLAGS} -gencode arch=compute_62,code=sm_62")
ENDIF()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants