ImportError: libcudart.so.10.1: cannot open shared object file: No such file or directory

[IISCAditayTripathi]

🐛 Bug

To Reproduce

Steps to reproduce the behavior:

1. from torchvision import _C

from torchvision import _C Traceback (most recent call last): File "", line 1, in ImportError: libcudart.so.10.1: cannot open shared object file: No such file or directory.

Environment

python collect_env.py

Collecting environment information...
PyTorch version: 1.1.0
Is debug build: False
CUDA used to build PyTorch: 10.0.130
ROCM used to build PyTorch: N/A
OS: Ubuntu 16.04.5 LTS (x86_64)
GCC version: (Ubuntu 8.4.0-1ubuntu1~16.04.1) 8.4.0
Clang version: Could not collect
CMake version: version 3.14.4
Python version: 3.6 (64-bit runtime)
Is CUDA available: True
CUDA runtime version: 10.0.130
GPU models and configuration:
GPU 0: GeForce GTX 1080 Ti
GPU 1: GeForce GTX 1080 Ti
GPU 2: GeForce GTX 1080 Ti
GPU 3: GeForce GTX 1080 Ti
GPU 4: GeForce GTX 1080 Ti
GPU 5: GeForce GTX 1080 Ti
GPU 6: GeForce GTX 1080 Ti
GPU 7: GeForce GTX 1080 Ti
Nvidia driver version: 418.39
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.7.5.0
/usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudnn.so.5.1.10
HIP runtime version: N/A
MIOpen runtime version: N/A
Versions of relevant libraries:
[pip3] numpy==1.19.5
[pip3] torch==1.1.0
[pip3] torchvision==0.4.2
[conda] cudatoolkit               10.0.130             hf841e97_6    conda-forge
[conda] mkl                       2020.2                      256
[conda] numpy                     1.19.5           py36h2aa4a07_1    conda-forge
[conda] pytorch                   1.1.0           py3.6_cuda10.0.130_cudnn7.5.1_0    pytorch
[conda] torchvision               0.3.0           py36_cu10.0.130_1    pytorch

Additional context

I was using fasterRCNN Object detector in torchvision while doing keep = nms(boxes_for_nms, scores, iou_threshold) it is giving this error. Easy way to reproduce this error is to run

from torchvision import _C

Please help.

cc @fmassa @vfdev-5

[imaginary-person]

You need CUDA Toolkit 10.1. Your system has CUDA Toolkit 10.0

[ngimel]

Please file an issue to torchvision

[IISCAditayTripathi]

You need CUDA Toolkit 10.1. Your system has CUDA Toolkit 10.0

@imaginary-person One another person's account on the same machine, this works. That person has CUDA Toolkit 10.0 installed and in his account, the import works.

[vfdev-5]

@IISCAditayTripathi I think you installed environment can be buggy:

• you have 2 versions of torchvision installed: 0.3.0 and 0.4.2

Currently, the latest version of torchvision is 0.8.2. If you would like the latest version, please, follow on how to install the latest stable version of torchvision with conda (https://pytorch.org/get-started/locally/):

conda install pytorch torchvision cudatoolkit=10.1 -c pytorch

(make sure to use a clean environment or uninstall previous installations).

[IISCAditayTripathi]

@vfdev-5 It is working now. The problem was there were two installations of torchvision one was compiled for CUDA 10.1 . After removing that version, it seems to be working. Thanks.

[fisakhan]

I have cuda 11 and get the following error. What should I do?

File "/home/twentyface/.virtualenvs/torch/lib/python3.6/site-packages/pytorch3d/renderer/blending.py", line 9, in <module>
    from pytorch3d import _C
ImportError: libcudart.so.10.1: cannot open shared object file: No such file or directory

sys.platform: linux
Python: 3.6.9 (default, Jan 26 2021, 15:33:00) [GCC 8.4.0]
CUDA available: True
GPU 0: NVIDIA GeForce GTX 1060
CUDA_HOME: /usr/local/cuda-11.0
NVCC: Build cuda_11.0_bu.TC445_37.28845127_0
GCC: gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
PyTorch: 1.6.0
PyTorch compiling details: PyTorch built with:
  - GCC 7.3
  - C++ Version: 201402
  - Intel(R) Math Kernel Library Version 2019.0.5 Product Build 20190808 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v1.5.0 (Git Hash e2ac1fac44c5078ca927cb9b90e1b3066a0b2ed0)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 10.2
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75
  - CuDNN 7.6.5
  - Magma 2.5.2
  - Build settings: BLAS=MKL, BUILD_TYPE=Release, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DUSE_VULKAN_WRAPPER -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, USE_CUDA=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_STATIC_DISPATCH=OFF, 

TorchVision: 0.7.0
OpenCV: 4.4.0
MMCV: 1.3.3
MMCV Compiler: GCC 7.5
MMCV CUDA Compiler: 11.0
MMDetection: 2.7.0+f5c8dc6

[fmassa]

@fisakhan the error seems to be within PyTorch3d, it would be better to report this error in https://github.com/facebookresearch/pytorch3d

[fisakhan]

Thanks @fmassa