[Caffe2] cudnn mismatch

[wjp0408]

❓ Questions and Help

Hi, Everyone.

This is my system config:

cuda: 8.0 cudnn: 6.0.21

Environment : A virtual environment, with

python2.7 in Anaconda2, cudatoolkit: 8.0, cudnn: 7.1.3

When I install caffe2 from source code, and run codes based on it, i met following issue:

And When I build, I check the cuda and cudnn information like this:

And i met these warning infos at the last:

This is the command I used in cmake:

cmake .. -DCMAKE_PREFIX_PATH=/home/pci/anaconda2/envs/spotter -DCMAKE_INSTALL_PREFIX=/home/pci/anaconda2/envs/spotter -DPYTHON_LIBRARY=/home/pci/anaconda2/envs/spotter/lib/python2.7/site-packages -DPYTHON_INCLUDE_DIR=/home/pci/anaconda2/envs/spotter/include/python2.7
-DUSE_NCCL=OFF

It shows that caffe2 builded with cuda8 and cudnn7.1 in build information, But...

So Why this happend...? And I want to know how to fix it...

Thank you very much !

[peterjc123]

I guess that you'll need cudatoolkit-dev for building PyTorch.

[wjp0408]

I guess that you'll need cudatoolkit-dev for building PyTorch.
Thanks for your quick reply !
And I'm sorry about that I'm new here ... I don't know how to use cudatoolkit-dev when building PyTorch... Do I need to change some options when Cmake? Can you tell me more about it...? Thanks...

[peterjc123]

conda install -c conda-forge cudatoolkit-dev

[wjp0408]

conda install -c conda-forge cudatoolkit-dev

Thanks for explaining that....
But I got this error when I use this command...

[peterjc123]

Sorry, it was late yesterday so I didn't have too much time to explain it clearly.
This error occured because you have system-level cuda and cudnn installed, and then you want to override that with one cudnn in the virtual environment. While the right cudnn library got chosen, it may not choose the right header for cudnn. To resolve this problem quickly, I suggested you use the cuda in the virtual environment, too. That is conda install -c conda-forge cudatoolkit-dev. But it seems the post link script of that library doesn't support python 2.7 well. As a user, the workground here is to uninstall/delete the system-level cudnn files(lib/include) and then it will be fine. For the devs, I think it will be better to add the variable CUDNN_HEADER_PATH to make it easy for the users to override the system-level cudnn. cc: @pjh5

[wjp0408]

Sorry, it was late yesterday so I didn't have too much time to explain it clearly.
This error occured because you have system-level cuda and cudnn installed, and then you want to override that with one cudnn in the virtual environment. While the right cudnn library got chosen, it may not choose the right header for cudnn. To resolve this problem quickly, I suggested you use the cuda in the virtual environment, too. That is conda install -c conda-forge cudatoolkit-dev. But it seems the post link script of that library doesn't support python 2.7 well. As a user, the workground here is to uninstall/delete the system-level cudnn files(lib/include) and then it will be fine. For the devs, I think it will be better to add the variable CUDNN_HEADER_PATH to make it easy for the users to override the system-level cudnn. cc: @pjh5

Thanks for your patience and help !
I follow your suggests (delete the system-level cudnn files) and solved it.
Have a nice day.：）

[pjh5]

cc @kostmo