how to install the NVIDIA’s deep learning stack on Windows: CUDA toolkit + cuDNN + TensorRT,
also various attempts to compile deep learning libraries with cuda
only for Windows 10/11 + recent RTX cards recommended
all comands here are for cmd, if u prefer pwsh must change to adapt
some values are censored with black box ███ so u have to fill in with your use case
1st of all: 👉👉👉 install CUDA toolkit + cuDNN + TensorRT 👈👈👈
my attempts to build various deep learning libraries:
- build onnxruntime ✅
- build whisper.cpp with cuBLAS ✅
- build CTranslate2 to use faster-whisper ❌
- build triton to use xformers ❌
- try out TensorRT-LLM ❌
- build OpenCV with CUDA ✅
- build Torch-TensorRT ❌
all above guides are under CC-BY-4.0
more complicated tweaking with TensorRT (under GPLv3)
- build Real-ESRGAN with TensorRT on Windows: https://github.com/phineas-pta/RealESRGAN-trt-win ✅
- run Stable Diffusion XL with TensorRT natively on Windows: https://github.com/phineas-pta/SDXL-trt-win ✅