After the new release CTranslate2 4.0.0, I'm no longer able to run on Windows a WhisperX model on GPU due to a CTranslate2 error: RuntimeError: Library cublas64_12.dll is not found or cannot be loaded.
Traceback (most recent call last):
File "D:\repos\whisper-gui\test.py", line 10, in <module>
res = model.transcribe(audio, print_progress=True)
File "C:\Users\pikur\miniconda3\envs\whisperx\lib\site-packages\whisperx\asr.py", line 194, in transcribe
language = language or self.detect_language(audio)
File "C:\Users\pikur\miniconda3\envs\whisperx\lib\site-packages\whisperx\asr.py", line 252, in detect_language
encoder_output = self.model.encode(segment)
File "C:\Users\pikur\miniconda3\envs\whisperx\lib\site-packages\whisperx\asr.py", line 86, in encode
return self.model.encode(features, to_cpu=to_cpu)
RuntimeError: Library cublas64_12.dll is not found or cannot be loaded
Despite the error appearing after calling a whisperx function, you can see the error is directly caused by the line self.model.encode(features, to_cpu=to_cpu), where self.model is a ctranslate2.models.Whisper(). Besides, neither whisperx nor faster-whisper packages have had significant releases these last days other than updating their requirements to CTranslate2 4.0.0.
Reproducibility
The python code:
import whisperx
model = whisperx.load_model("base", "cuda", compute_type="int8", download_root="models/whisperx")
audio = whisperx.load_audio("examples/coffe_break_example.mp3")
res = model.transcribe(audio, print_progress=True)
print(res)
I'm in a Conda environment with Python 3.10, Pytorch 2.0.0 and pytorch-cuda 11.8.
Environment packages
# Name Version Build Channel
aiofiles 23.2.1 pypi_0 pypi
aiohttp 3.9.3 pypi_0 pypi
aiosignal 1.3.1 pypi_0 pypi
alembic 1.13.1 pypi_0 pypi
altair 5.2.0 pypi_0 pypi
annotated-types 0.6.0 pypi_0 pypi
antlr4-python3-runtime 4.9.3 pypi_0 pypi
anyio 4.3.0 pypi_0 pypi
asteroid-filterbanks 0.4.0 pypi_0 pypi
async-timeout 4.0.3 pypi_0 pypi
attrs 23.2.0 pypi_0 pypi
audioread 3.0.1 pypi_0 pypi
av 11.0.0 pypi_0 pypi
blas 1.0 mkl
bzip2 1.0.8 he774522_0
ca-certificates 2023.12.12 haa95532_0
certifi 2024.2.2 pypi_0 pypi
cffi 1.16.0 pypi_0 pypi
charset-normalizer 3.3.2 pypi_0 pypi
click 8.1.7 pypi_0 pypi
colorama 0.4.6 pypi_0 pypi
coloredlogs 15.0.1 pypi_0 pypi
colorlog 6.8.2 pypi_0 pypi
contourpy 1.2.0 pypi_0 pypi
ctranslate2 4.0.0 pypi_0 pypi
cuda-cccl 12.3.101 0 nvidia
cuda-cudart 11.8.89 0 nvidia
cuda-cudart-dev 11.8.89 0 nvidia
cuda-cupti 11.8.87 0 nvidia
cuda-libraries 11.8.0 0 nvidia
cuda-libraries-dev 11.8.0 0 nvidia
cuda-nvrtc 11.8.89 0 nvidia
cuda-nvrtc-dev 11.8.89 0 nvidia
cuda-nvtx 11.8.86 0 nvidia
cuda-profiler-api 12.3.101 0 nvidia
cuda-runtime 11.8.0 0 nvidia
cycler 0.12.1 pypi_0 pypi
decorator 5.1.1 pypi_0 pypi
docopt 0.6.2 pypi_0 pypi
einops 0.7.0 pypi_0 pypi
exceptiongroup 1.2.0 pypi_0 pypi
fastapi 0.110.0 pypi_0 pypi
faster-whisper 1.0.0 pypi_0 pypi
ffmpy 0.3.2 pypi_0 pypi
filelock 3.13.1 py310haa95532_0
flatbuffers 23.5.26 pypi_0 pypi
fonttools 4.49.0 pypi_0 pypi
frozenlist 1.4.1 pypi_0 pypi
fsspec 2024.2.0 pypi_0 pypi
gmpy2 2.1.2 py310h7f96b67_0
gradio 4.16.0 pypi_0 pypi
gradio-client 0.8.1 pypi_0 pypi
greenlet 3.0.3 pypi_0 pypi
h11 0.14.0 pypi_0 pypi
httpcore 1.0.4 pypi_0 pypi
httpx 0.27.0 pypi_0 pypi
huggingface-hub 0.20.3 pypi_0 pypi
humanfriendly 10.0 pypi_0 pypi
hyperpyyaml 1.2.2 pypi_0 pypi
idna 3.6 pypi_0 pypi
importlib-resources 6.1.2 pypi_0 pypi
intel-openmp 2023.1.0 h59b6b97_46320
jinja2 3.1.3 py310haa95532_0
joblib 1.3.2 pypi_0 pypi
jsonschema 4.21.1 pypi_0 pypi
jsonschema-specifications 2023.12.1 pypi_0 pypi
julius 0.2.7 pypi_0 pypi
kiwisolver 1.4.5 pypi_0 pypi
lazy-loader 0.3 pypi_0 pypi
libcublas 11.11.3.6 0 nvidia
libcublas-dev 11.11.3.6 0 nvidia
libcufft 10.9.0.58 0 nvidia
libcufft-dev 10.9.0.58 0 nvidia
libcurand 10.3.4.107 0 nvidia
libcurand-dev 10.3.4.107 0 nvidia
libcusolver 11.4.1.48 0 nvidia
libcusolver-dev 11.4.1.48 0 nvidia
libcusparse 11.7.5.86 0 nvidia
libcusparse-dev 11.7.5.86 0 nvidia
libffi 3.4.4 hd77b12b_0
libnpp 11.8.0.86 0 nvidia
libnpp-dev 11.8.0.86 0 nvidia
libnvjpeg 11.9.0.86 0 nvidia
libnvjpeg-dev 11.9.0.86 0 nvidia
librosa 0.10.1 pypi_0 pypi
libuv 1.44.2 h2bbff1b_0
lightning 2.2.0.post0 pypi_0 pypi
lightning-utilities 0.10.1 pypi_0 pypi
llvmlite 0.42.0 pypi_0 pypi
mako 1.3.2 pypi_0 pypi
markdown-it-py 3.0.0 pypi_0 pypi
markupsafe 2.1.3 py310h2bbff1b_0
matplotlib 3.8.3 pypi_0 pypi
mdurl 0.1.2 pypi_0 pypi
mkl 2023.1.0 h6b88ed4_46358
mkl-service 2.4.0 py310h2bbff1b_1
mkl_fft 1.3.8 py310h2bbff1b_0
mkl_random 1.2.4 py310h59b6b97_0
mpc 1.1.0 h7edee0f_1
mpfr 4.0.2 h62dcd97_1
mpir 3.0.0 hec2e145_1
mpmath 1.3.0 py310haa95532_0
msgpack 1.0.7 pypi_0 pypi
multidict 6.0.5 pypi_0 pypi
networkx 3.1 py310haa95532_0
nltk 3.8.1 pypi_0 pypi
numba 0.59.0 pypi_0 pypi
numpy 1.26.4 py310h055cbcc_0
numpy-base 1.26.4 py310h65a83cf_0
omegaconf 2.3.0 pypi_0 pypi
onnxruntime 1.17.1 pypi_0 pypi
openssl 3.0.13 h2bbff1b_0
optuna 3.5.0 pypi_0 pypi
orjson 3.9.15 pypi_0 pypi
packaging 23.2 pypi_0 pypi
pandas 2.2.1 pypi_0 pypi
pillow 10.2.0 pypi_0 pypi
pip 23.3.1 py310haa95532_0
platformdirs 4.2.0 pypi_0 pypi
pooch 1.8.1 pypi_0 pypi
primepy 1.3 pypi_0 pypi
protobuf 4.25.3 pypi_0 pypi
pyannote-audio 3.1.1 pypi_0 pypi
pyannote-core 5.0.0 pypi_0 pypi
pyannote-database 5.0.1 pypi_0 pypi
pyannote-metrics 3.2.1 pypi_0 pypi
pyannote-pipeline 3.0.1 pypi_0 pypi
pycparser 2.21 pypi_0 pypi
pydantic 2.6.2 pypi_0 pypi
pydantic-core 2.16.3 pypi_0 pypi
pydub 0.25.1 pypi_0 pypi
pygments 2.17.2 pypi_0 pypi
pyparsing 3.1.1 pypi_0 pypi
pyreadline3 3.4.1 pypi_0 pypi
python 3.10.13 he1021f5_0
python-dateutil 2.8.2 pypi_0 pypi
python-multipart 0.0.9 pypi_0 pypi
pytorch 2.0.0 py3.10_cuda11.8_cudnn8_0 pytorch
pytorch-cuda 11.8 h24eeafa_5 pytorch
pytorch-lightning 2.2.0.post0 pypi_0 pypi
pytorch-metric-learning 2.4.1 pypi_0 pypi
pytorch-mutex 1.0 cuda pytorch
pytz 2024.1 pypi_0 pypi
pyyaml 6.0.1 pypi_0 pypi
referencing 0.33.0 pypi_0 pypi
regex 2023.12.25 pypi_0 pypi
requests 2.31.0 pypi_0 pypi
rich 13.7.0 pypi_0 pypi
rpds-py 0.18.0 pypi_0 pypi
ruamel-yaml 0.18.6 pypi_0 pypi
ruamel-yaml-clib 0.2.8 pypi_0 pypi
ruff 0.2.2 pypi_0 pypi
safetensors 0.4.2 pypi_0 pypi
scikit-learn 1.4.1.post1 pypi_0 pypi
scipy 1.12.0 pypi_0 pypi
semantic-version 2.10.0 pypi_0 pypi
semver 3.0.2 pypi_0 pypi
sentencepiece 0.2.0 pypi_0 pypi
setuptools 68.2.2 py310haa95532_0
shellingham 1.5.4 pypi_0 pypi
six 1.16.0 pypi_0 pypi
sniffio 1.3.1 pypi_0 pypi
sortedcontainers 2.4.0 pypi_0 pypi
soundfile 0.12.1 pypi_0 pypi
soxr 0.3.7 pypi_0 pypi
speechbrain 0.5.16 pypi_0 pypi
sqlalchemy 2.0.27 pypi_0 pypi
sqlite 3.41.2 h2bbff1b_0
starlette 0.36.3 pypi_0 pypi
sympy 1.12 py310haa95532_0
tabulate 0.9.0 pypi_0 pypi
tbb 2021.8.0 h59b6b97_0
tensorboardx 2.6.2.2 pypi_0 pypi
threadpoolctl 3.3.0 pypi_0 pypi
tk 8.6.12 h2bbff1b_0
tokenizers 0.15.2 pypi_0 pypi
tomlkit 0.12.0 pypi_0 pypi
toolz 0.12.1 pypi_0 pypi
torch-audiomentations 0.11.1 pypi_0 pypi
torch-pitch-shift 1.2.4 pypi_0 pypi
torchaudio 2.0.0 pypi_0 pypi
torchmetrics 1.3.1 pypi_0 pypi
tqdm 4.66.2 pypi_0 pypi
transformers 4.38.1 pypi_0 pypi
typer 0.9.0 pypi_0 pypi
typing_extensions 4.9.0 py310haa95532_1
tzdata 2024.1 pypi_0 pypi
urllib3 2.2.1 pypi_0 pypi
uvicorn 0.27.1 pypi_0 pypi
vc 14.2 h21ff451_1
vs2015_runtime 14.27.29016 h5e58377_2
websockets 11.0.3 pypi_0 pypi
wheel 0.41.2 py310haa95532_0
whisperx 3.1.1 pypi_0 pypi
xz 5.4.5 h8cc25b3_0
yarl 1.9.4 pypi_0 pypi
zlib 1.2.13 h8cc25b3_0
nvcc --version output in terminal:
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:41:10_Pacific_Daylight_Time_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0
As I said, this error only happens on Windows, I've been able to run the same exact script, with the same model, audio, dependencies, etc. on Linux (even with a lower cuda version) without problems.
I hope you can fix this issue, thank you in advance.
After the new release CTranslate2 4.0.0, I'm no longer able to run on Windows a WhisperX model on GPU due to a CTranslate2 error:
RuntimeError: Library cublas64_12.dll is not found or cannot be loaded.Despite the error appearing after calling a
whisperxfunction, you can see the error is directly caused by the lineself.model.encode(features, to_cpu=to_cpu), whereself.modelis actranslate2.models.Whisper(). Besides, neitherwhisperxnorfaster-whisperpackages have had significant releases these last days other than updating their requirements to CTranslate2 4.0.0.Reproducibility
The python code:
I'm in a Conda environment with Python 3.10, Pytorch 2.0.0 and pytorch-cuda 11.8.
Environment packages
nvcc --versionoutput in terminal:As I said, this error only happens on Windows, I've been able to run the same exact script, with the same model, audio, dependencies, etc. on Linux (even with a lower cuda version) without problems.
I hope you can fix this issue, thank you in advance.