bitsandbytes was compiled without GPU support. 8-bit optimizers and GPU quantization are unavailable.

[vjeronymo2]

Hi, I'm trying to use the 8-bit optimizer with an A100 on a OpenPBS environment, which means that the machine I install the python virtual environment, with the bitsandbytes lib, doesn't have the GPU, but when I submit the job it does.
Hence, I'm getting this error:
The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers and GPU quantization are unavailable.

Which makes me think it's because I installed the library on the machine without GPU. How can I install it with GPU support on a machine without GPU? I'm using cu116 btw

PS.When I submit the job, the target machine doesn't have access to internet, so it's not able to fetch the library or anything else online.

Thanks in advance

[TimDettmers]

The binary that is used is determined at runtime. This means in your case there are two modes of failures:

• the CUDA driver is not detected (libcuda.so)
• the runtime library is not detected (libcudart.so)

Both libraries need to be detected in order to find the right library for the GPU/CUDA version that you are trying to execute against.

Could you please provide use with the output of python -m bitsandbytes for the GPU job that you are trying to run?

[TemporalLabsLLC-SOL]

I think I'm in a similiar boat trying to get this to engage with the python version of dreambooth-sd

[aicrumb]

hi, same issue, exact same use-case too haha

===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
For effortless bug reporting copy-paste your error into this form: https://docs.google.com/forms/d/e/1FAIpQLScPB8emS3Thkp66nvqwmjTEgxp8Y9ufuWTzFyr9kJ5AoI47dQ/viewform?usp=sf_link
================================================================================
/opt/conda/lib/python3.7/site-packages/bitsandbytes/cuda_setup/paths.py:21: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/nvidia/lib'), PosixPath('/usr/local/nvidia/lib64')}
  "WARNING: The following directories listed in your path were found to "
/opt/conda/lib/python3.7/site-packages/bitsandbytes/cuda_setup/paths.py:99: UserWarning: /usr/local/nvidia/lib:/usr/local/nvidia/lib64 did not contain libcudart.so as expected! Searching further paths...
  f'{candidate_env_vars["LD_LIBRARY_PATH"]} did not contain '
/opt/conda/lib/python3.7/site-packages/bitsandbytes/cuda_setup/paths.py:21: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('module'), PosixPath('//matplotlib_inline.backend_inline')}
  "WARNING: The following directories listed in your path were found to "
/opt/conda/lib/python3.7/site-packages/bitsandbytes/cuda_setup/paths.py:21: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('noninteractive    SHELL=/bin/bash')}
  "WARNING: The following directories listed in your path were found to "
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64...
/opt/conda/lib/python3.7/site-packages/bitsandbytes/cuda_setup/paths.py:21: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/cuda/lib64')}
  "WARNING: The following directories listed in your path were found to "
WARNING: No libcudart.so found! Install CUDA or the cudatoolkit package (anaconda)!
CUDA SETUP: Loading binary /opt/conda/lib/python3.7/site-packages/bitsandbytes/libbitsandbytes_cpu.so...
/opt/conda/lib/python3.7/site-packages/bitsandbytes/cextension.py:49: UserWarning: The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers and GPU quantization are unavailable.
  "The installed version of bitsandbytes was compiled without GPU support. "

I know Im using a cuda-enabled machine

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.85.02    Driver Version: 510.85.02    CUDA Version: 11.6     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  On   | 00000000:01:00.0 Off |                  N/A |
| 30%   31C    P8    20W / 350W |      1MiB / 24576MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

[TimDettmers]

I believe this is fixed in the latest version. It prints instructions on how to debug the situation and alternatively prints out compilation instructions which should fix the issue.

[aykutkilic-avl]

I had the same issue, (nvidia-smi had proper output, but was still getting this error)

1. I've downloaded cuda drivers from nvidia's website, and followed their instructions:

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-12-0_12.0.1-1_amd64.deb
sudo dpkg -i cuda-12-0_12.0.1-1_amd64.deb

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-ubuntu2204.pin
sudo mv cuda-ubuntu2204.pin /etc/apt/preferences.d/cuda-repository-pin-600

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-drivers_525.85.12-1_amd64.deb
wget https://developer.download.nvidia.com/compute/cuda/repos/$distro/$arch/cuda-keyring_1.0-1_all.deb
sudo dpkg -i cuda-keyring_1.0-1_all.deb
sudo apt-key del 7fa2af80

sudo apt-get install cuda

2. it failed and messed up my debian package graph, (I've done a bunch of below, it will be different on other systems but just to document)

sudo apt --fix-broken install
sudo systemctl stop nvidia-powerd
sudo apt-get remove nvidia-kernel-common-515
sudo dpkg -r nvidia-compute-utils-525
sudo dpkg -i --force-overwrite /var/cache/apt/archives/nvidia-compute-utils-525_525.85.12-0ubuntu1_amd64.deb
sudo dpkg --configure -a
sudo apt-get install -f

3. Once everything is sorted out, rebooted the system:

sudo reboot

Now it works.

[swumagic]

Bitsandbytes was not supported windows before, but my method can support windows.（yuhuang）
1 open folder J:\StableDiffusion\sdwebui，Click the address bar of the folder and enter CMD
or WIN+R, CMD 。enter，cd /d J:\StableDiffusion\sdwebui
2 J:\StableDiffusion\sdwebui\py310\python.exe -m pip uninstall bitsandbytes

3 J:\StableDiffusion\sdwebui\py310\python.exe -m pip uninstall bitsandbytes-windows

4 J:\StableDiffusion\sdwebui\py310\python.exe -m pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.41.1-py3-none-win_amd64.whl

Replace your SD venv directory file（python.exe Folder） here（J:\StableDiffusion\sdwebui\py310）