"CUDA Setup failed despite GPU being available" - when running python -m bitsandbytes

[Asrith-Dasari-AuxoAI]

False

===================================BUG REPORT===================================
/anaconda/envs/azureml_py38/lib/python3.8/site-packages/bitsandbytes-0.41.1-py3.8.egg/bitsandbytes/cuda_setup/main.py:166: UserWarning: Welcome to bitsandbytes. For bug reports, please run

python -m bitsandbytes

warn(msg)

/anaconda/envs/azureml_py38/lib/python3.8/site-packages/bitsandbytes-0.41.1-py3.8.egg/bitsandbytes/cuda_setup/main.py:166: UserWarning: /anaconda/envs/azureml_py38 did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths...
warn(msg)
The following directories listed in your path were found to be non-existent: {PosixPath('/opt/intel/compilers_and_libraries_2018.3.222/linux/mpi/mic/lib'), PosixPath('/usr/local/cuda/extras/CUPTI/lib64')}
/anaconda/envs/azureml_py38/lib/python3.8/site-packages/bitsandbytes-0.41.1-py3.8.egg/bitsandbytes/cuda_setup/main.py:166: UserWarning: Found duplicate ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] files: {PosixPath('/usr/local/cuda/lib64/libcudart.so'), PosixPath('/usr/local/cuda/lib64/libcudart.so.11.0'), PosixPath('/anaconda/envs/azureml_py38/lib/libcudart.so')}.. We select the PyTorch default libcudart.so, which is {torch.version.cuda},but this might missmatch with the CUDA version that is needed for bitsandbytes.To override this behavior set the BNB_CUDA_VERSION=<version string, e.g. 122> environmental variableFor example, if you want to use the CUDA version 122BNB_CUDA_VERSION=122 python ...OR set the environmental variable in your .bashrc: export BNB_CUDA_VERSION=122In the case of a manual override, make sure you set the LD_LIBRARY_PATH, e.g.export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-11.2
warn(msg)
/anaconda/envs/azureml_py38/lib/python3.8/site-packages/bitsandbytes-0.41.1-py3.8.egg/bitsandbytes/cuda_setup/main.py:166: UserWarning: /opt/intel/compilers_and_libraries_2018.3.222/linux/mpi/intel64/lib:/opt/intel/compilers_and_libraries_2018.3.222/linux/mpi/mic/lib:/opt/intel/compilers_and_libraries_2018.3.222/linux/mpi/intel64/lib:/opt/intel/compilers_and_libraries_2018.3.222/linux/mpi/mic/lib::/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64/:/anaconda/envs/azureml_py38/lib/:/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64/:/anaconda/envs/azureml_py38/lib/ did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths...
warn(msg)
The following directories listed in your path were found to be non-existent: {PosixPath('-DCMAKE_AR=/anaconda/envs/azureml_py38/bin/x86_64-conda-linux-gnu-ar -DCMAKE_CXX_COMPILER_AR=/anaconda/envs/azureml_py38/bin/x86_64-conda-linux-gnu-gcc-ar -DCMAKE_C_COMPILER_AR=/anaconda/envs/azureml_py38/bin/x86_64-conda-linux-gnu-gcc-ar -DCMAKE_RANLIB=/anaconda/envs/azureml_py38/bin/x86_64-conda-linux-gnu-ranlib -DCMAKE_CXX_COMPILER_RANLIB=/anaconda/envs/azureml_py38/bin/x86_64-conda-linux-gnu-gcc-ranlib -DCMAKE_C_COMPILER_RANLIB=/anaconda/envs/azureml_py38/bin/x86_64-conda-linux-gnu-gcc-ranlib -DCMAKE_LINKER=/anaconda/envs/azureml_py38/bin/x86_64-conda-linux-gnu-ld -DCMAKE_STRIP=/anaconda/envs/azureml_py38/bin/x86_64-conda-linux-gnu-strip')}
The following directories listed in your path were found to be non-existent: {PosixPath('8888'), PosixPath('http'), PosixPath('//localhost')}
The following directories listed in your path were found to be non-existent: {PosixPath('-fvisibility-inlines-hidden -std=c++17 -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /anaconda/envs/azureml_py38/include')}
The following directories listed in your path were found to be non-existent: {PosixPath('-fvisibility-inlines-hidden -std=c++17 -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-all -fno-plt -Og -g -Wall -Wextra -fvar-tracking-assignments -ffunction-sections -pipe -isystem /anaconda/envs/azureml_py38/include')}
The following directories listed in your path were found to be non-existent: {PosixPath('-Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,-rpath,/anaconda/envs/azureml_py38/lib -Wl,-rpath-link,/anaconda/envs/azureml_py38/lib -L/anaconda/envs/azureml_py38/lib')}
The following directories listed in your path were found to be non-existent: {PosixPath('//eastus.api.azureml.ms/mlflow/v1.0/subscriptions/d0e6d200-b258-43fc-b02e-46e3b60eadda/resourceGroups/lama_2_expt/providers/Microsoft.MachineLearningServices/workspaces/lama2'), PosixPath('azureml')}
The following directories listed in your path were found to be non-existent: {PosixPath('-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-all -fno-plt -Og -g -Wall -Wextra -fvar-tracking-assignments -ffunction-sections -pipe -isystem /anaconda/envs/azureml_py38/include')}
The following directories listed in your path were found to be non-existent: {PosixPath('-DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /anaconda/envs/azureml_py38/include')}
The following directories listed in your path were found to be non-existent: {PosixPath('//127.0.0.1'), PosixPath('46808/OBO/token'), PosixPath('http')}
The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/cuda/extras/CUPTI/lib64')}
The following directories listed in your path were found to be non-existent: {PosixPath('-D_DEBUG -D_FORTIFY_SOURCE=2 -Og -isystem /anaconda/envs/azureml_py38/include')}
The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/cuda/extras/CUPTI/lib64')}
The following directories listed in your path were found to be non-existent: {PosixPath('/usr/lib/node_modules')}
The following directories listed in your path were found to be non-existent: {PosixPath('//127.0.0.1'), PosixPath('46808/MSI/auth'), PosixPath('http')}
The following directories listed in your path were found to be non-existent: {PosixPath('-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /anaconda/envs/azureml_py38/include')}
/anaconda/envs/azureml_py38/lib/python3.8/site-packages/bitsandbytes-0.41.1-py3.8.egg/bitsandbytes/cuda_setup/main.py:166: UserWarning: Found duplicate ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] files: {PosixPath('/usr/local/cuda/lib64/libcudart.so'), PosixPath('/usr/local/cuda/lib64/libcudart.so.11.0')}.. We select the PyTorch default libcudart.so, which is {torch.version.cuda},but this might missmatch with the CUDA version that is needed for bitsandbytes.To override this behavior set the BNB_CUDA_VERSION=<version string, e.g. 122> environmental variableFor example, if you want to use the CUDA version 122BNB_CUDA_VERSION=122 python ...OR set the environmental variable in your .bashrc: export BNB_CUDA_VERSION=122In the case of a manual override, make sure you set the LD_LIBRARY_PATH, e.g.export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-11.2
warn(msg)
DEBUG: Possible options found for libcudart.so: {PosixPath('/usr/local/cuda/lib64/libcudart.so'), PosixPath('/usr/local/cuda/lib64/libcudart.so.11.0')}
CUDA SETUP: CUDA version lower than 11 are currently not supported for LLM.int8(). You will be only to use 8-bit optimizers and quantization routines!!
CUDA SETUP: PyTorch settings found: CUDA_VERSION=102, Highest Compute Capability: 3.7.
CUDA SETUP: To manually override the PyTorch CUDA version please see:https://github.com/TimDettmers/bitsandbytes/blob/main/how_to_use_nonpytorch_cuda.md
/anaconda/envs/azureml_py38/lib/python3.8/site-packages/bitsandbytes-0.41.1-py3.8.egg/bitsandbytes/cuda_setup/main.py:166: UserWarning: WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU! If you run into issues with 8-bit matmul, you can try 4-bit quantization: https://huggingface.co/blog/4bit-transformers-bitsandbytes
warn(msg)
CUDA SETUP: Required library version not found: libbitsandbytes_cuda102_nocublaslt.so. Maybe you need to compile it from source?
CUDA SETUP: Defaulting to libbitsandbytes_cpu.so...

================================================ERROR=====================================
CUDA SETUP: CUDA detection failed! Possible reasons:

1. You need to manually override the PyTorch CUDA version. Please see: "https://github.com/TimDettmers/bitsandbytes/blob/main/how_to_use_nonpytorch_cuda.md
2. CUDA driver not installed
3. CUDA not installed
4. You have multiple conflicting CUDA libraries
5. Required library not pre-compiled for this bitsandbytes release!
   CUDA SETUP: If you compiled from source, try again with make CUDA_VERSION=DETECTED_CUDA_VERSION for example, make CUDA_VERSION=113.
   CUDA SETUP: The CUDA version for the compile might depend on your conda install. Inspect CUDA version via conda list | grep cuda.
   ================================================================================

CUDA SETUP: Something unexpected happened. Please compile from source:
git clone https://github.com/TimDettmers/bitsandbytes.git
cd bitsandbytes
CUDA_VERSION=102_nomatmul
python setup.py install
CUDA SETUP: Setup Failed!
Traceback (most recent call last):
File "/anaconda/envs/azureml_py38/lib/python3.8/runpy.py", line 185, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "/anaconda/envs/azureml_py38/lib/python3.8/runpy.py", line 144, in _get_module_details
return _get_module_details(pkg_main_name, error)
File "/anaconda/envs/azureml_py38/lib/python3.8/runpy.py", line 111, in _get_module_details
import(pkg_name)
File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/bitsandbytes-0.41.1-py3.8.egg/bitsandbytes/init.py", line 6, in
from . import cuda_setup, utils, research
File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/bitsandbytes-0.41.1-py3.8.egg/bitsandbytes/research/init.py", line 1, in
from . import nn
File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/bitsandbytes-0.41.1-py3.8.egg/bitsandbytes/research/nn/init.py", line 1, in
from .modules import LinearFP8Mixed, LinearFP8Global
File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/bitsandbytes-0.41.1-py3.8.egg/bitsandbytes/research/nn/modules.py", line 8, in
from bitsandbytes.optim import GlobalOptimManager
File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/bitsandbytes-0.41.1-py3.8.egg/bitsandbytes/optim/init.py", line 6, in
from bitsandbytes.cextension import COMPILED_WITH_CUDA
File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/bitsandbytes-0.41.1-py3.8.egg/bitsandbytes/cextension.py", line 20, in
raise RuntimeError('''
RuntimeError:
CUDA Setup failed despite GPU being available. Please run the following command to get more information:

    python -m bitsandbytes

    Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them
    to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes
    and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues

[TristaCheng2018]

have you solved this problem?

[Asrith-Dasari-AuxoAI]

This issue wasn't solved

[therealcyberlord]

running into the same error

[sudofeng]

running into the same error

+1

[Nguyendat-bit]

@TimDettmers can u fix this problem ?

[nonelight]

re-install bitsandbytes with these,make sure to set cuda version to your cuda:
`pip uninstall bitsandbytes
git clone https://github.com/TimDettmers/bitsandbytes.git
cd bitsandbytes
make CUDA_VERSION=117 #your CUDA_VERSION
python setup.py install```

[Asrith-Dasari-AuxoAI]

re-install bitsandbytes with these,make sure to set cuda version to your cuda: `pip uninstall bitsandbytes git clone https://github.com/TimDettmers/bitsandbytes.git cd bitsandbytes make CUDA_VERSION=117 #your CUDA_VERSION python setup.py install```

Just wanted to confirm if by CUDA_VERSION you mean is the value that we get by doing this torch.version.cuda?

[Asrith-Dasari-AuxoAI]

re-install bitsandbytes with these,make sure to set cuda version to your cuda: `pip uninstall bitsandbytes git clone https://github.com/TimDettmers/bitsandbytes.git cd bitsandbytes make CUDA_VERSION=117 #your CUDA_VERSION python setup.py install```

I have done this but it didnt help.
Any other cuda optimization library that we can use instead of bitsandbytes?

[nonelight]

re-install bitsandbytes with these,make sure to set cuda version to your cuda: `pip uninstall bitsandbytes git clone https://github.com/TimDettmers/bitsandbytes.git cd bitsandbytes make CUDA_VERSION=117 #your CUDA_VERSION python setup.py install```

Just wanted to confirm if by CUDA_VERSION you mean is the value that we get by doing this torch.version.cuda?

nvcc -V show the cuda version

[Asrith-Dasari-AuxoAI]

nvcc -V

I did what you have told but its still giving me the same error

[nonelight]

nvcc -V

I did what you have told but its still giving me the same error

ubuntu version,GPU ,cuda version,pytorch version ,these must be matched. show your info please. and confirm how many cuda in your system

[Asrith-Dasari-AuxoAI]

nvcc -V

I did what you have told but its still giving me the same error

ubuntu version,GPU ,cuda version,pytorch version ,these must be matched. show your info please. and confirm how many cuda in your system

Will it be possible for us to connect over a call?

[cateyelow]

1. Download bitsandbytes-0.39.1-py3-none-win_amd64.whl

2. cd [Download Directory]
   pip install bitsandbytes-0.39.1-py3-none-win_amd64.whl

3. remove this code in C:\Users\UserName\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes_main_.py

    if 'CONDA_PREFIX' in os.environ:
        paths = find_file_recursive(os.environ['CONDA_PREFIX'], f'*cuda*{SHARED_LIB_EXTENSION}')
        print_header("ANACONDA CUDA PATHS")
        print(paths)
        print('')
    if 'CUDA_HOME' in os.environ:
        paths = find_file_recursive(os.environ['CONDA_PREFIX'], f'*cuda*{SHARED_LIB_EXTENSION}')
        print_header("CUDA_HOME CUDA PATHS")
        print(paths)
        print('')
    elif 'CUDA_PATH' in os.environ:
        paths = find_file_recursive(os.environ['CONDA_PREFIX'], f'*cuda*{SHARED_LIB_EXTENSION}')
        print_header("CUDA_PATH CUDA PATHS")
        print(paths)
        print('')

4. python -m bitsandbytes

C:\Users\Master>python -m bitsandbytes

===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please run

python -m bitsandbytes

 and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
================================================================================
bin C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda117.dll
CUDA SETUP: CUDA runtime path found: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\bin\cudart64_110.dll
CUDA SETUP: Highest compute capability among GPUs detected: 8.6
CUDA SETUP: Detected CUDA version 117
CUDA SETUP: Loading binary C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda117.dll...
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++ BUG REPORT INFORMATION ++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

+++++++++++++++ WORKING DIRECTORY CUDA PATHS +++++++++++++++
C:\Users\Master\anaconda3\envs\stylegan3\bin\cudart32_110.dll
C:\Users\Master\anaconda3\envs\stylegan3\bin\cudart64_110.dll
C:\Users\Master\anaconda3\envs\stylegan3\lib\site-packages\torch\lib\c10_cuda.dll
C:\Users\Master\anaconda3\envs\stylegan3\lib\site-packages\torch\lib\torch_cuda.dll
C:\Users\Master\anaconda3\envs\stylegan3\Library\bin\cudart64_110.dll
C:\Users\Master\anaconda3\pkgs\cuda-cudart-dev-11.8.89-0\bin\cudart32_110.dll
C:\Users\Master\anaconda3\pkgs\cuda-cudart-dev-11.8.89-0\bin\cudart64_110.dll
C:\Users\Master\anaconda3\pkgs\cudatoolkit-11.1.1-hb074779_11\Library\bin\cudart64_110.dll
C:\Users\Master\anaconda3\pkgs\cudatoolkit-11.8.0-h09e9e62_11\Library\bin\cudart64_110.dll
C:\Users\Master\anaconda3\pkgs\pytorch-2.0.1-py3.9_cuda11.8_cudnn8_0\Lib\site-packages\torch\lib\c10_cuda.dll
C:\Users\Master\anaconda3\pkgs\pytorch-2.0.1-py3.9_cuda11.8_cudnn8_0\Lib\site-packages\torch\lib\torch_cuda.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda111.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda111_nocublaslt.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda112.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda112_nocublaslt.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda113.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda113_nocublaslt.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda114.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda114_nocublaslt.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda115.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda115_nocublaslt.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda116.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda116_nocublaslt.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda117.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda117_nocublaslt.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda118.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda118_nocublaslt.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda120.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda120_nocublaslt.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda121.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda121_nocublaslt.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda122.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda122_nocublaslt.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\cuda_setup\libbitsandbytes_cuda116.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\lib\c10_cuda.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\lib\cudart64_110.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\lib\torch_cuda.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\torchvision\cudart64_110.dll
C:\Users\Master\AppData\Local\Programs\Python\Python37\Lib\site-packages\torch\lib\c10_cuda.dll
C:\Users\Master\AppData\Local\Programs\Python\Python37\Lib\site-packages\torch\lib\cudart64_110.dll
C:\Users\Master\AppData\Local\Programs\Python\Python37\Lib\site-packages\torch\lib\torch_cuda.dll
C:\Users\Master\AppData\Local\Programs\Python\Python37\Lib\site-packages\torch\lib\torch_cuda_cpp.dll
C:\Users\Master\AppData\Local\Programs\Python\Python37\Lib\site-packages\torch\lib\torch_cuda_cu.dll
C:\Users\Master\AppData\Local\Programs\Python\Python37\Lib\site-packages\torchvision\cudart64_110.dll
C:\Users\Master\Downloads\libbitsandbytes_cuda116.dll

++++++++++++++++++ LD_LIBRARY CUDA PATHS +++++++++++++++++++
 C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\lib\x64 CUDA PATHS
Could not read LD_LIBRARY_PATH: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\lib\x64

++++++++++++++++++++++++++ OTHER +++++++++++++++++++++++++++
COMPILED_WITH_CUDA = True
COMPUTE_CAPABILITIES_PER_GPU = ['8.6']
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++ DEBUG INFO END ++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Running a quick check that:
    + library is importable
    + CUDA function is callable


WARNING: Please be sure to sanitize sensible info from any such env vars!

SUCCESS!
Installation was successful!

[alongLFB]

1. Download bitsandbytes-0.39.1-py3-none-win_amd64.whl
2. cd [Download Directory]
   pip install bitsandbytes-0.39.1-py3-none-win_amd64.whl
3. remove this code in C:\Users\UserName\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes__main__.py

    if 'CONDA_PREFIX' in os.environ:
        paths = find_file_recursive(os.environ['CONDA_PREFIX'], f'*cuda*{SHARED_LIB_EXTENSION}')
        print_header("ANACONDA CUDA PATHS")
        print(paths)
        print('')
    if 'CUDA_HOME' in os.environ:
        paths = find_file_recursive(os.environ['CONDA_PREFIX'], f'*cuda*{SHARED_LIB_EXTENSION}')
        print_header("CUDA_HOME CUDA PATHS")
        print(paths)
        print('')
    elif 'CUDA_PATH' in os.environ:
        paths = find_file_recursive(os.environ['CONDA_PREFIX'], f'*cuda*{SHARED_LIB_EXTENSION}')
        print_header("CUDA_PATH CUDA PATHS")
        print(paths)
        print('')

4. python -m bitsandbytes

C:\Users\Master>python -m bitsandbytes

===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please run

python -m bitsandbytes

 and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
================================================================================
bin C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda117.dll
CUDA SETUP: CUDA runtime path found: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\bin\cudart64_110.dll
CUDA SETUP: Highest compute capability among GPUs detected: 8.6
CUDA SETUP: Detected CUDA version 117
CUDA SETUP: Loading binary C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda117.dll...
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++ BUG REPORT INFORMATION ++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

+++++++++++++++ WORKING DIRECTORY CUDA PATHS +++++++++++++++
C:\Users\Master\anaconda3\envs\stylegan3\bin\cudart32_110.dll
C:\Users\Master\anaconda3\envs\stylegan3\bin\cudart64_110.dll
C:\Users\Master\anaconda3\envs\stylegan3\lib\site-packages\torch\lib\c10_cuda.dll
C:\Users\Master\anaconda3\envs\stylegan3\lib\site-packages\torch\lib\torch_cuda.dll
C:\Users\Master\anaconda3\envs\stylegan3\Library\bin\cudart64_110.dll
C:\Users\Master\anaconda3\pkgs\cuda-cudart-dev-11.8.89-0\bin\cudart32_110.dll
C:\Users\Master\anaconda3\pkgs\cuda-cudart-dev-11.8.89-0\bin\cudart64_110.dll
C:\Users\Master\anaconda3\pkgs\cudatoolkit-11.1.1-hb074779_11\Library\bin\cudart64_110.dll
C:\Users\Master\anaconda3\pkgs\cudatoolkit-11.8.0-h09e9e62_11\Library\bin\cudart64_110.dll
C:\Users\Master\anaconda3\pkgs\pytorch-2.0.1-py3.9_cuda11.8_cudnn8_0\Lib\site-packages\torch\lib\c10_cuda.dll
C:\Users\Master\anaconda3\pkgs\pytorch-2.0.1-py3.9_cuda11.8_cudnn8_0\Lib\site-packages\torch\lib\torch_cuda.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda111.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda111_nocublaslt.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda112.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda112_nocublaslt.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda113.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda113_nocublaslt.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda114.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda114_nocublaslt.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda115.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda115_nocublaslt.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda116.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda116_nocublaslt.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda117.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda117_nocublaslt.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda118.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda118_nocublaslt.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda120.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda120_nocublaslt.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda121.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda121_nocublaslt.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda122.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\libbitsandbytes_cuda122_nocublaslt.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\bitsandbytes\cuda_setup\libbitsandbytes_cuda116.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\lib\c10_cuda.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\lib\cudart64_110.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\lib\torch_cuda.dll
C:\Users\Master\AppData\Local\Programs\Python\Python311\Lib\site-packages\torchvision\cudart64_110.dll
C:\Users\Master\AppData\Local\Programs\Python\Python37\Lib\site-packages\torch\lib\c10_cuda.dll
C:\Users\Master\AppData\Local\Programs\Python\Python37\Lib\site-packages\torch\lib\cudart64_110.dll
C:\Users\Master\AppData\Local\Programs\Python\Python37\Lib\site-packages\torch\lib\torch_cuda.dll
C:\Users\Master\AppData\Local\Programs\Python\Python37\Lib\site-packages\torch\lib\torch_cuda_cpp.dll
C:\Users\Master\AppData\Local\Programs\Python\Python37\Lib\site-packages\torch\lib\torch_cuda_cu.dll
C:\Users\Master\AppData\Local\Programs\Python\Python37\Lib\site-packages\torchvision\cudart64_110.dll
C:\Users\Master\Downloads\libbitsandbytes_cuda116.dll

++++++++++++++++++ LD_LIBRARY CUDA PATHS +++++++++++++++++++
 C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\lib\x64 CUDA PATHS
Could not read LD_LIBRARY_PATH: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\lib\x64

++++++++++++++++++++++++++ OTHER +++++++++++++++++++++++++++
COMPILED_WITH_CUDA = True
COMPUTE_CAPABILITIES_PER_GPU = ['8.6']
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++ DEBUG INFO END ++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Running a quick check that:
    + library is importable
    + CUDA function is callable


WARNING: Please be sure to sanitize sensible info from any such env vars!

SUCCESS!
Installation was successful!

this is for windows, how to solve this problem in centos? need downgrade cuda?

[Aniwine]

I solved it by updating torch from 1.13.1 to 2.0.1 and torchvison from 0.14.1 to 0.15.2.