cuDNN 5.1 not working - CUDNN_STATUS_BAD_PARAM

[duduRodrigues]

Issue summary

Basically, i am having a low performance in both computers i've tried to run the openpose. I first have tried on a computer with 32gb ram, gtx 960 4gb and a core i7-4790K. I got 3.4 fps running. Then i tried to run it on a pc with 64gb o ram, 3xGtx 780ti sli and a Core i7-5930K @ 3.50GHz × 12 and got 7~10 fps.
I take a look at the Q: Low Speed and try to downgrade my cuDNN to the 5.1 version (i was using 6v) and compile the code, but when i try to run the openpose i keep getting this error:

Starting pose estimation demo.
Auto-detecting GPUs... Detected 3 GPU(s), using them all.
Starting thread(s)
F0609 16:51:49.077610 8598 cudnn.hpp:122] Check failed: status == CUDNN_STATUS_SUCCESS (3 vs. 0) CUDNN_STATUS_BAD_PARAM
*** Check failure stack trace: ***
@ 0x7f6aa1e955cd google::LogMessage::Fail()
@ 0x7f6aa1e97433 google::LogMessage::SendToLog()
@ 0x7f6aa1e9515b google::LogMessage::Flush()
@ 0x7f6aa1e97e1e google::LogMessageFatal::~LogMessageFatal()
@ 0x7f6aa08acc78 caffe::CuDNNConvolutionLayer<>::Reshape()
@ 0x7f6aa09c9b92 caffe::Net<>::Init()
@ 0x7f6aa09cc9b0 caffe::Net<>::Net()
@ 0x7f6aa21b143e op::NetCaffe::initializationOnThread()
@ 0x7f6aa21e272f op::PoseExtractorCaffe::netInitializationOnThread()
@ 0x7f6aa2135c51 op::SubThread<>::initializationOnThread()
@ 0x7f6aa2136240 op::Thread<>::initializationOnThread()
@ 0x7f6aa213b0ad op::Thread<>::threadFunction()
@ 0x7f6aa199fc80 (unknown)
@ 0x7f6a9efd36ba start_thread
@ 0x7f6aa140e82d clone
@ (nil) (unknown)
Aborted (core dumped)

What is this? How can i fix it?

Your system configuration

Ubuntu 16.04
Cuda 8.0
cuDNN 6
Core i7-5930K @ 3.50GHz × 12
3xGTX 780 Ti
64gb RAM
Default
OpenCV 2.4.9.1
gcc 5.4.0

[gineshidalgo99]

Since you changed cuDNN, you have to re-compile Caffe and OpenPose (and basically any other non-related program that you compiled with cuDNN).
https://github.com/CMU-Perceptual-Computing-Lab/openpose/blob/master/doc/installation.md#reinstallation

(Brute force alternative: delete and re-download OpenPose)

About speed, considering the current benchmark (I added your 960 speed), it is the expected speed for those graphics cards with the default setting (you can always speed it up with the Low Speed tips):
Benchmark: https://docs.google.com/spreadsheets/d/1-DynFGvoScvfWDA1P4jDInCkbD4lg0IKOYbXgEq0sK0/edit#gid=0

[duduRodrigues]

To get things working i've compiled Caffe and OpenPose with cuDNN 6 installed on usr/local/cuda/ and then changed manually my cuDNN 6 files to the cuDNN 5.1 on usr/local/cuda/. Do you think that the compiled OpenPose is running with cuDNN 5.1 or cuDNN 6?

[gineshidalgo99]

It is using 5 thinking that is 6, therefore the error. So you have to recompile OpenPose/Caffe.