CUDA_SETUP: WARNING! libcudart.so not found in any environmental path

[Shedarin]

I US WINDOWS, NOT LINUX . I am 100% sure that the problem is not in kohya-ss/sd-scripts. and not in https://github.com/derrian-distro/LoRA_Easy_Training_Scripts. I tried to train LoRA with this script https://github.com/derrian-distro/LoRA_Easy_Training_Scripts but got an error:

CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64...
C:\Artem\ai\SD-вещи\kohya-ss-sd-scripts\sd-scripts\venv\lib\site-packages\bitsandbytes\cuda_setup\paths.py:27: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('/usr/local/cuda/lib64')}
warn(
WARNING: No libcudart.so found! Install CUDA or the cudatoolkit package (anaconda)!
CUDA SETUP: Loading binary C:\Artem\ai\SD-вещи\kohya-ss-sd-scripts\sd-scripts\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cudaall.so...
Traceback (most recent call last):
File "C:\Artem\ai\SD-вещи\kohya-ss-sd-scripts\sd-scripts\lora_train_popup.py", line 535, in
main()
File "C:\Artem\ai\SD-вещи\kohya-ss-sd-scripts\sd-scripts\lora_train_popup.py", line 203, in main
train_network.train(args)
File "C:\Artem\ai\SD-вещи\kohya-ss-sd-scripts\sd-scripts\train_network.py", line 134, in train
import bitsandbytes as bnb
File "C:\Artem\ai\SD-вещи\kohya-ss-sd-scripts\sd-scripts\venv\lib\site-packages\bitsandbytes_init_.py", line 6, in
from .autograd._functions import (
File "C:\Artem\ai\SD-вещи\kohya-ss-sd-scripts\sd-scripts\venv\lib\site-packages\bitsandbytes\autograd_functions.py", line 5, in
import bitsandbytes.functional as F
File "C:\Artem\ai\SD-вещи\kohya-ss-sd-scripts\sd-scripts\venv\lib\site-packages\bitsandbytes\functional.py", line 13, in
from .cextension import COMPILED_WITH_CUDA, lib
File "C:\Artem\ai\SD-вещи\kohya-ss-sd-scripts\sd-scripts\venv\lib\site-packages\bitsandbytes\cextension.py", line 43, in
lib = CUDALibrary_Singleton.get_instance().lib
File "C:\Artem\ai\SD-вещи\kohya-ss-sd-scripts\sd-scripts\venv\lib\site-packages\bitsandbytes\cextension.py", line 39, in get_instance
cls.instance.initialize()
File "C:\Artem\ai\SD-вещи\kohya-ss-sd-scripts\sd-scripts\venv\lib\site-packages\bitsandbytes\cextension.py", line 33, in initialize
self.lib = ct.cdll.LoadLibrary(str(binary_path)) # $$$
File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\ctypes_init.py", line 452, in LoadLibrary
return self.dlltype(name)
File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\ctypes_init.py", line 374, in init
self._handle = _dlopen(self._name, mode)
OSError: [WinError 193] %1 ist keine zulässige Win32-Anwendung
Traceback (most recent call last):
File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "C:\Artem\ai\SD-вещи\kohya-ss-sd-scripts\sd-scripts\venv\Scripts\accelerate.exe_main.py", line 7, in
File "C:\Artem\ai\SD-вещи\kohya-ss-sd-scripts\sd-scripts\venv\lib\site-packages\accelerate\commands\accelerate_cli.py", line 45, in main
args.func(args)
File "C:\Artem\ai\SD-вещи\kohya-ss-sd-scripts\sd-scripts\venv\lib\site-packages\accelerate\commands\launch.py", line 1104, in launch_command
simple_launcher(args)
File "C:\Artem\ai\SD-вещи\kohya-ss-sd-scripts\sd-scripts\venv\lib\site-packages\accelerate\commands\launch.py", line 567, in simple_launcher
raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['C:\Artem\ai\SD-вещи\kohya-ss-sd-scripts\sd-scripts\venv\Scripts\python.exe', 'lora_train_popup.py']' returned non-zero exit status 1.

BUT I HAVE ALREASY downloaded CUDA, I had uninstalled CUDA 12 and downloaded version 11.6 and cuDNN v8.7.0 but it still didn't help. I also have anaconda installed.

As far as I know /usr/local/cuda/lib64 and libcudart.so are somehow related to Linux, but I have Windows

the command:
git clone https://github.com/TimDettmers/bitsandbytes.git
cd bitsandbytes
CUDA_VERSION=120 make cuda11x
python setup.py install
does not help because, command CUDA_VERSION=120 make cuda11x does not exist.

This method won't help because it's DAMN LINUX:
LD_LIBRARY_PATH=/usr/local/cuda/lib64/:$LD_LIBRARY_PATH

P.S. this method doesn't help either: https://github.com/james-things/bitsandbytes-prebuilt-all_arch

and I repeat again, I have WINDOWS, NOT LINUX

[etherealxx]

i got this too when running on colab https://colab.research.google.com/github/Linaqruf/kohya-trainer/blob/main/kohya-LoRA-dreambooth.ipynb

[abhinavkulkarni]

I also get this with the latest versions of bitsandbytes, accelerate and transformers. Simply trying to import bitsandbytes results in this error.

Is the library meant to work on Windows

[Northloop]

@Shedarin @etherealxx @abhinavkulkarni has anyone found anything that works for this in the last two weeks?

[etherealxx]

@Shedarin @etherealxx @abhinavkulkarni has anyone found anything that works for this in the last two weeks?

Mine works. Tried just yesterday.

[SaltyBarnacles]

Has anyone found a fix?

[deviandice]

@SaltyBarnacles @Northloop

I fixed this by making the following modifications in to the main.py script of setupcuda. This works in WSL.
Simply replace the functions in cudaSetup/main.py replace with these, it just searches at a specified location (you'll need to provide the path)

One of the main issues with this is it doesn't search Conda, Miniconda or WSL folders where it usually is. Dunno why lol.

def check_cuda_result(cuda, result_val):
    # 3. Check for CUDA errors
    if result_val != 0:
        error_str = ctypes.c_char_p()
        cuda.cuGetErrorString(result_val, ctypes.byref(error_str))
        print(f"CUDA exception! Error code: {error_str.value.decode()}")

def get_cuda_version(cuda, cudart_path):
    # https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART____VERSION.html#group__CUDART____VERSION
    try:
        cudart = ctypes.CDLL(cudart_path)
		try:
            cuda = ctypes.CDLL("path to libcudart.so") #  Change the path to be the exact location of libcudart.so
    	except OSError:
        	# TODO: shouldn't we error or at least warn here?
        	print(f'ERROR: libcudart.so could not be read from path: {cudart_path}!')
        	return None

    version = ctypes.c_int()
    check_cuda_result(cuda, cudart.cudaRuntimeGetVersion(ctypes.byref(version)))
    version = int(version.value)
    major = version//1000
    minor = (version-(major*1000))//10

    if major < 11:
       print('CUDA SETUP: CUDA version lower than 11 are currenlty not supported for LLM.int8(). You will be only to use 8-bit optimizers and quantization routines!!')

    return f'{major}{minor}'

def get_cuda_lib_handle():
    # 1. find libcuda.so library (GPU driver) (/usr/lib)
    try:
        cuda = ctypes.CDLL("libcuda.so")
    except OSError:
        # 2. try looking in alternate path (/usr/lib/wsl/lib/libcuda.so)
        try:
            cuda = ctypes.CDLL("path to libcuda.so") #  Change the path to be the exact location of libcuda.so
        except OSError:
            # TODO: shouldn't we error or at least warn here?
            print('CUDA SETUP: WARNING! libcuda.so not found! Do you have a CUDA driver installed? If you are on a cluster, make sure you are on a CUDA machine!')
            return None
    check_cuda_result(cuda, cuda.cuInit(0))

    return cuda

Find the paths using these commands

cd /
sudo find -iname "libcudart.so" 
sudo find -iname "libcuda.so"

[wdhameli0]

i fixed this by replacing 'libcuda.so' to 'libbitsandbytes_cuda118.so' in main.py of cuda_setup.

[LLLYF]

我通过在cuda_setup main.py 中将“libcuda.so”替换为“libbitsandbytes_cuda118.so”来解决此问

Excuse me sir , could please tell me how to check the name that own myself.

[wdhameli0]

我通过在cuda_setup main.py 中将“libcuda.so”替换为“libbitsandbytes_cuda118.so”来解决此问

Excuse me sir , could please tell me how to check the name that own myself.

You could check .../Python/Lib/site-packages/bitsandbytes, in this directory, you can find the "libbitsandbytes_cuda(XXX).so". Then get into /cuda_setup, open the main.py. Find "def get_cuda_lib_handle() , replacing libcuda.so to libbitsandbytes_cuda(XXX).so

[LLLYF]

我通过在cuda_setup main.py 中将“libcuda.so”替换为“libbitsandbytes_cuda118.so”来解决此问

Excuse me sir , could please tell me how to check the name that own myself.

You could check .../Python/Lib/site-packages/bitsandbytes, in this directory, you can find the "libbitsandbytes_cuda(XXX).so". Then get into /cuda_setup, open the main.py. Find "def get_cuda_lib_handle() , replacing libcuda.so to libbitsandbytes_cuda(XXX).so

Thanks sir! I just got them, and I change the name like "libbitsandbytes_cuda117.so", 'libbitsandbytes_cuda118.so', 'libbitsandbytes_cuda120.so' but it does not work. My cuda is 12.1

[wdhameli0]

我通过在cuda_setup main.py 中将“libcuda.so”替换为“libbitsandbytes_cuda118.so”来解决此问

Excuse me sir , could please tell me how to check the name that own myself.

You could check .../Python/Lib/site-packages/bitsandbytes, in this directory, you can find the "libbitsandbytes_cuda(XXX).so". Then get into /cuda_setup, open the main.py. Find "def get_cuda_lib_handle() , replacing libcuda.so to libbitsandbytes_cuda(XXX).so

Thanks sir! I just got them, and I change the name like "libbitsandbytes_cuda117.so", 'libbitsandbytes_cuda118.so', 'libbitsandbytes_cuda120.so' but it does not work. My cuda is 12.1

I found this problem while using stable diffusion webui. My cuda is 12.1 too, but in stable diffusion webui ,it's

[wdhameli0]

我通过在cuda_setup main.py 中将“libcuda.so”替换为“libbitsandbytes_cuda118.so”来解决此问

Excuse me sir , could please tell me how to check the name that own myself.

You could check .../Python/Lib/site-packages/bitsandbytes, in this directory, you can find the "libbitsandbytes_cuda(XXX).so". Then get into /cuda_setup, open the main.py. Find "def get_cuda_lib_handle() , replacing libcuda.so to libbitsandbytes_cuda(XXX).so

Thanks sir! I just got them, and I change the name like "libbitsandbytes_cuda117.so", 'libbitsandbytes_cuda118.so', 'libbitsandbytes_cuda120.so' but it does not work. My cuda is 12.1

after replacing 'libcuda.so' to 'libbitsandbytes_cuda118.so',

[LLLYF]

我通过在cuda_setup main.py 中将“libcuda.so”替换为“libbitsandbytes_cuda118.so”来解决此问

Excuse me sir , could please tell me how to check the name that own myself.

You could check .../Python/Lib/site-packages/bitsandbytes, in this directory, you can find the "libbitsandbytes_cuda(XXX).so". Then get into /cuda_setup, open the main.py. Find "def get_cuda_lib_handle() , replacing libcuda.so to libbitsandbytes_cuda(XXX).so

Thanks sir! I just got them, and I change the name like "libbitsandbytes_cuda117.so", 'libbitsandbytes_cuda118.so', 'libbitsandbytes_cuda120.so' but it does not work. My cuda is 12.1

I found this problem while using stable diffusion webui. My cuda is 12.1 too, but in stable diffusion webui ,it's

我通过在cuda_setup main.py 中将“libcuda.so”替换为“libbitsandbytes_cuda118.so”来解决此问

Excuse me sir , could please tell me how to check the name that own myself.

You could check .../Python/Lib/site-packages/bitsandbytes, in this directory, you can find the "libbitsandbytes_cuda(XXX).so". Then get into /cuda_setup, open the main.py. Find "def get_cuda_lib_handle() , replacing libcuda.so to libbitsandbytes_cuda(XXX).so

Thanks sir! I just got them, and I change the name like "libbitsandbytes_cuda117.so", 'libbitsandbytes_cuda118.so', 'libbitsandbytes_cuda120.so' but it does not work. My cuda is 12.1

after replacing 'libcuda.so' to 'libbitsandbytes_cuda118.so',

Stable diffusion should not change these files... While I change them as you did, the same problem was came out.

我通过在cuda_setup main.py 中将“libcuda.so”替换为“libbitsandbytes_cuda118.so”来解决此问

Excuse me sir , could please tell me how to check the name that own myself.

You could check .../Python/Lib/site-packages/bitsandbytes, in this directory, you can find the "libbitsandbytes_cuda(XXX).so". Then get into /cuda_setup, open the main.py. Find "def get_cuda_lib_handle() , replacing libcuda.so to libbitsandbytes_cuda(XXX).so

Thanks sir! I just got them, and I change the name like "libbitsandbytes_cuda117.so", 'libbitsandbytes_cuda118.so', 'libbitsandbytes_cuda120.so' but it does not work. My cuda is 12.1

after replacing 'libcuda.so' to 'libbitsandbytes_cuda118.so',

Stable diffusion should not change these files... While I change them as you did, the same problem was came out.

[cebtenzzre]

I am on Arch Linux, and I am not using anaconda. I have libcudart.so in /opt/cuda/targets/x86_64-linux/lib, but bitsandbytes searches LD_LIBRARY_PATH even though the path is only added via /etc/ld.so.conf.d/cuda.conf. Why can't it fall back to a simple ctypes.CDLL('libcudart.so'), which works fine?