Check failed: status == CUDNN_STATUS_SUCCESS (1 vs. 0) CUDNN_STATUS_NOT_INITIALIZED #2166
Comments
FaterYU
changed the title
|
I have the same problem, did you solve it? |
Not yet. I am still trying to find a way. |
|
Same issue. Did you find a solution? |
1 similar comment
|
Same issue. Did you find a solution? |
|
Running into the same problem, I tried uninstalling everything (nvidia drivers/cuda/cudnn) and reinstalled again with the recomendations on the prerequisites page (CUDA 11.7 and Cudnn 8.5.0 since I am in ubuntu 20.04) but still runinng on the same error. In my case, the first time it throws this error: My setup: |
|
It worked for me a little bit, but it also has some errors: F0227 13:43:18.177312 7780 syncedmem.cpp:71] Check failed: error == cudaSuccess (2 vs. 0) out of memory
*** Check failure stack trace: ***
@ 0x7fc1d4ce11c3 google::LogMessage::Fail()
@ 0x7fc1d4ce625b google::LogMessage::SendToLog()
@ 0x7fc1d4ce0ebf google::LogMessage::Flush()
@ 0x7fc1d4ce16ef google::LogMessageFatal::~LogMessageFatal()
@ 0x7fc1d4a3259a caffe::SyncedMemory::mutable_gpu_data()
@ 0x7fc1d48cb336 caffe::Blob<>::mutable_gpu_data()
@ 0x7fc1d4908b60 caffe::BaseConvolutionLayer<>::forward_gpu_gemm()
@ 0x7fc1d4a6bac1 caffe::ConvolutionLayer<>::Forward_gpu()
@ 0x7fc1d49f22d2 caffe::Net<>::ForwardFromTo()
@ 0x7fc1d55a38f7 op::NetCaffe::forwardPass()
@ 0x7fc1d55878e7 op::detectHandKeypoints()
@ 0x7fc1d5588489 op::HandExtractorCaffe::forwardPass()
@ 0x7fc1d558355f op::WHandExtractorNet<>::work()
@ 0x7fc1d55e80df op::Worker<>::checkAndWork()
@ 0x7fc1d55e826b op::SubThread<>::workTWorkers()
@ 0x7fc1d55f5ddd op::SubThreadQueueInOut<>::work()
@ 0x7fc1d55ece41 op::Thread<>::threadFunction()
@ 0x7fc1d55ed1e5 op::Thread<>::exec()
@ 0x7fc1d55efea4 op::ThreadManager<>::exec()
@ 0x7fc1d5645819 op::WrapperT<>::exec()
@ 0x55c60140422c openPoseDemo()
@ 0x55c6014006c7 main
@ 0x7fc1d4f54083 __libc_start_main
@ 0x55c6014006fe _start
[1] 7780 abort ./build/examples/openpose/openpose.bin --video examples/media/video.aviDo you know if my memory of GPU is not enough? My GPU is MX450 with 2GB memory. |
I think 2GB is not enough for AI processing. Even with my GPU (8GB), I sometimes have memory problems. What I can suggest you is lowering the resolution of the image or video you want to process. Try with a really small image (something like 254x254) and see if that works for you. Then increase the resolution until you find a threshhold for your purpose. Also, try to close background processes as much as possible. Other solutions would require you editing the source code, so if you just want to test this, lowering the resolution would be the fastest approach. |
|
It works for me. The error is about my GPU memory. |
Issue Summary
When I run
./build/examples/openpose/openpose.bin --video examples/media/video.avi --face --hand --write_json output_json_folder/, it raises some error likecuDNN did not return a suitable algorithm for convolution.. But cudnn actually exists in system.Executed Command
Errors (if any)
Type of Issue
Your System Configuration
Whole console output https://pastebin.com/dPb5i8b7
OpenPose version: Latest GitHub code
General configuration:
Non-default settings:
3rd-party software:
cmake --versionin Ubuntu):apt-get install libopencv-dev(only Ubuntu); OpenPose default (only Windows); compiled from source? If so, 2.4.9, 2.4.12, 3.1, 3.2?; ...?If GPU mode issue:
cat /usr/local/cuda/version.txtin most cases):{ "cuda" : { "name" : "CUDA SDK", "version" : "11.7.1" }, "cuda_cccl" : { "name" : "CUDA C++ Core Compute Libraries", "version" : "11.7.91" }, "cuda_cudart" : { "name" : "CUDA Runtime (cudart)", "version" : "11.7.99" }, "cuda_cuobjdump" : { "name" : "cuobjdump", "version" : "11.7.91" }, "cuda_cupti" : { "name" : "CUPTI", "version" : "11.7.101" }, "cuda_cuxxfilt" : { "name" : "CUDA cu++ filt", "version" : "11.7.91" }, "cuda_demo_suite" : { "name" : "CUDA Demo Suite", "version" : "11.7.91" }, "cuda_gdb" : { "name" : "CUDA GDB", "version" : "11.7.91" }, "cuda_memcheck" : { "name" : "CUDA Memcheck", "version" : "11.7.91" }, "cuda_nsight" : { "name" : "Nsight Eclipse Plugins", "version" : "11.7.91" }, "cuda_nvcc" : { "name" : "CUDA NVCC", "version" : "11.7.99" }, "cuda_nvdisasm" : { "name" : "CUDA nvdisasm", "version" : "11.7.91" }, "cuda_nvml_dev" : { "name" : "CUDA NVML Headers", "version" : "11.7.91" }, "cuda_nvprof" : { "name" : "CUDA nvprof", "version" : "11.7.101" }, "cuda_nvprune" : { "name" : "CUDA nvprune", "version" : "11.7.91" }, "cuda_nvrtc" : { "name" : "CUDA NVRTC", "version" : "11.7.99" }, "cuda_nvtx" : { "name" : "CUDA NVTX", "version" : "11.7.91" }, "cuda_nvvp" : { "name" : "CUDA NVVP", "version" : "11.7.101" }, "cuda_sanitizer_api" : { "name" : "CUDA Compute Sanitizer API", "version" : "11.7.91" }, "libcublas" : { "name" : "CUDA cuBLAS", "version" : "11.10.3.66" }, "libcufft" : { "name" : "CUDA cuFFT", "version" : "10.7.2.91" }, "libcufile" : { "name" : "GPUDirect Storage (cufile)", "version" : "1.3.1.18" }, "libcurand" : { "name" : "CUDA cuRAND", "version" : "10.2.10.91" }, "libcusolver" : { "name" : "CUDA cuSOLVER", "version" : "11.4.0.1" }, "libcusparse" : { "name" : "CUDA cuSPARSE", "version" : "11.7.4.91" }, "libnpp" : { "name" : "CUDA NPP", "version" : "11.7.4.75" }, "libnvjpeg" : { "name" : "CUDA nvJPEG", "version" : "11.8.0.2" }, "nsight_compute" : { "name" : "Nsight Compute", "version" : "2022.2.1.3" }, "nsight_systems" : { "name" : "Nsight Systems", "version" : "2022.1.3.3" }, "nvidia_fs" : { "name" : "NVIDIA file-system", "version" : "2.12.8" } }nvidia-smiin Ubuntu):The text was updated successfully, but these errors were encountered: