[Installation]: Blackwell SM120 + CUDA 13 pip install: 5 sequential failures before Qwen3.5 27B+ runs

[Gavin-Qiao]

Your current environment

Collecting environment information... uv is set
============================== System Info ============================== OS : Ubuntu 24.04.4 LTS (x86_64) GCC version : (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0 Clang version : Could not collect CMake version : Could not collect Libc version : glibc-2.39

     PyTorch Info

==============================
PyTorch version : 2.10.0+cu130
Is debug build : False
CUDA used to build PyTorch : 13.0 ROCM used to build PyTorch : N/A ==============================
Python Environment

Python version : 3.12.3 (main, Mar 3 2026, 12:15:18) [GCC 13.3.0] (64-bit runtime)
Python platform : Linux-6.6.87.2-microsoft-standard-WSL2-x86_64-with-glibc2.39
============================== CUDA / GPU Info
============================== Is CUDA available : True CUDA runtime version : Could not collect
CUDA_MODULE_LOADING set to :
GPU models and configuration : GPU 0: NVIDIA RTX PRO 5000 Blackwell
Nvidia driver version : 595.79 cuDNN version : Could not collect ==============================
CPU Info

Architecture: x86_64
Model name: Intel(R) Core(TM) Ultra 9 285K
CPU(s): 24 Hypervisor vendor: Microsoft ==============================
Versions of relevant libraries

[pip3] flashinfer-python==0.6.4
[pip3] numpy==2.2.6
[pip3] nvidia-cuda-runtime==13.0.96
[pip3] nvidia-cuda-runtime-cu12==12.9.79 [pip3] torch==2.10.0+cu130
[pip3] torchvision==0.25.0+cu130
[pip3] transformers==4.57.6
[pip3] triton==3.6.0 ============================== vLLM Info

vLLM Version : 0.17.1
vLLM Build Flags:
CUDA Archs: Not Set; ROCm: Disabled

How you are installing vllm

uv pip install vllm --torch-backend=auto

Summary

Setting up vLLM 0.17.1 from PyPI on a Blackwell GPU (RTX PRO 5000, SM120) with CUDA 13.0 on WSL2 requires solving 5 undocumented problems before any
Qwen3.5 27B+ model will run. Each failure produces a cryptic error with no actionable guidance. This issue documents the full chain and proposes fixes.

Problem 1: libcudart.so.12 not found

Error:

ImportError: libcudart.so.12: cannot open shared object file: No such file or directory

Root cause: vLLM's C extensions (_C.abi3.so, _moe_C.abi3.so, _vllm_fa2_C.abi3.so, _vllm_fa3_C.abi3.so, _flashmla_C.abi3.so,
_flashmla_extension_C.abi3.so) are compiled against CUDA 12 and link libcudart.so.12. CUDA 13 systems only ship libcudart.so.13.

Commonly suggested workaround: LD_LIBRARY_PATH. This is fragile — it doesn't survive process forks (vLLM's multiprocess EngineCore architecture),
systemd services, background tasks, or Docker entrypoints.

What actually worked: Installing nvidia-cuda-runtime-cu12 (provides libcudart.so.12) and then running patchelf --set-rpath on all 8 affected
.so files to bake in the path. No env vars needed.

Proposed fix: Either:

• (a) Add nvidia-cuda-runtime-cu12 as a dependency and set RPATH in vLLM's build system (CMakeLists.txt)
• (b) Publish versioned wheels (cu12, cu130) on PyPI like PyTorch does, with clear install instructions
• (c) At minimum, detect the mismatch at import time and print an actionable error message

Related: #30435, #31018, #28669, #35432

---

Problem 2: FlashInfer JIT requires undocumented system dependencies

FlashInfer JIT-compiles CUDA kernels at runtime for GDN/Mamba attention patterns (used by Qwen3.5 27B+). This requires four system packages that aren't
documented anywhere:

Missing dependency	Error message	Fix
gcc	RuntimeError: Failed to find C compiler	apt install gcc
python3.12-dev	gcc compilation fails (missing Python.h)	apt install python3.12-dev
CUDA toolkit (nvcc)	RuntimeError: Could not find nvcc and default cuda_home='/usr/local/cuda' doesn't exist	apt install cuda-toolkit
ninja	FileNotFoundError: [Errno 2] No such file or directory: 'ninja'	apt install ninja-build

Each failure is discovered sequentially — you fix one, hit the next. The errors come from deep inside FlashInfer's JIT layer, not vLLM, making them
hard to trace.

Proposed fix:

• Document these as system requirements in the installation guide, at least for Blackwell/GDN models
• Detect missing dependencies at startup and print a single actionable message: "FlashInfer JIT compilation requires: gcc, python3-dev, nvcc
  (cuda-toolkit), ninja-build"
• Consider pre-compiling FlashInfer kernels for common architectures (SM120, SM100) and shipping them in the wheel

Related: #21960, #32826

---

Problem 3: Qwen3.5 27B+ GDN models silently require --max-num-batched-tokens 2096

Error:

RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}

Actual root cause (buried in EngineCore subprocess log):

AssertionError: In Mamba cache align mode, block_size (X) must be <= max_num_batched_tokens

All Qwen3.5 models with GDN (Gated DeltaNet) layers — including the 27B dense model (not just MoE variants) — require --max-num-batched-tokens 2096
due to mamba/GDN cache alignment constraints. vLLM's default of 8192 violates this.

The 27B has 64 layers: 16 groups × (3 GDN + 1 Attention). vLLM treats GDN like Mamba for cache alignment, setting attention block size to 1568 tokens to ensure that attention page size is >= mamba page size.

Proposed fix:

• Auto-detect GDN models and cap max_num_batched_tokens at the mamba page size
• At minimum, surface the EngineCore subprocess error message to the user (see Problem 5)
• Document this in the Qwen3.5 recipe

Related: #36010, #35502

---

Problem 4: --attention-backend FLASH_ATTN + --kv-cache-dtype fp8 = silent crash

Error (again, buried in EngineCore subprocess):

ValueError: Selected backend AttentionBackendEnum.FLASH_ATTN is not valid for this configuration.
Reason: ['kv_cache_dtype not supported']

Users may set --attention-backend FLASH_ATTN as a workaround for FlashInfer crashes (#36828), not knowing it's incompatible with --kv-cache-dtype fp8.

Proposed fix:

• Validate flag combinations before spawning EngineCore subprocesses
• Raise a clear error: "FLASH_ATTN does not support kv-cache-dtype=fp8. Use the default FlashInfer backend or remove --kv-cache-dtype fp8."

Related: #12543, #35577, PR #14221

---

Problem 5: "Engine core initialization failed" provides no useful information

RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}

This error appears for every problem above (and many others — see #17618, #18730, #19002, #21882, #23176, #26898, #33245). The actual root cause is
always in the EngineCore subprocess log, which users must manually find at /tmp/vllm-*.log or by adding --log-level DEBUG.

I understand a logging redesign is tracked at #31683. In the meantime, a minimal fix would be:

• Capture the EngineCore subprocess's last exception and include it in the parent's error message
• Change Failed core proc(s): {} to actually list the subprocess PIDs and their exit codes/signals

---

Reproduction

# On any Blackwell GPU with CUDA 13.0 driver, WSL2 or native Linux
pip install vllm  # or: uv pip install vllm

# Problem 1: immediate crash
python -c "import vllm"
# ImportError: libcudart.so.12: cannot open shared object file

# After fixing libcudart (patchelf or LD_LIBRARY_PATH):
# Problem 2: crash on GDN model load (gcc/nvcc/ninja missing)
vllm serve Qwen/Qwen3.5-27B-FP8

# After installing system deps:
# Problem 3: crash with default --max-num-batched-tokens
vllm serve Qwen/Qwen3.5-27B-FP8 --kv-cache-dtype fp8
# Engine core initialization failed

# Problem 4: crash with explicit FLASH_ATTN + fp8
vllm serve Qwen/Qwen3.5-27B-FP8 --attention-backend FLASH_ATTN --kv-cache-dtype fp8
# Engine core initialization failed

# What actually works (after all fixes):
vllm serve Qwen/Qwen3.5-27B-FP8 --max-num-batched-tokens 2096 --kv-cache-dtype fp8 --dtype bfloat16

Suggested priority

Problems 1 and 5 are the highest impact — they affect every Blackwell user installing from PyPI, and the error messages give no path to resolution.
Problems 2-4 compound the frustration but are solvable once you know what to look for.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[Gavin-Qiao]

Complete workaround script

For anyone hitting these issues on Blackwell (SM120/SM121) + CUDA 13, here's everything in one place.

1. Project setup with uv

pyproject.toml:

[project]
name = "my-project"
requires-python = ">=3.12"
dependencies = [
    "vllm>=0.17.1",
    "torch>=2.10.0",
    "torchvision>=0.25.0",
    "nvidia-cuda-runtime-cu12",
]

[tool.uv.sources]
torch = [{ index = "pytorch-cu130", marker = "sys_platform == 'linux'" }]
torchvision = [{ index = "pytorch-cu130", marker = "sys_platform == 'linux'" }]

[[tool.uv.index]]
name = "pytorch-cu130"
url = "https://download.pytorch.org/whl/cu130"
explicit = true

Then:

uv sync

2. Fix libcudart.so.12 (patch RPATHs)

LD_LIBRARY_PATH is fragile — doesn't survive vLLM's multiprocess EngineCore forks. Patch the RPATHs directly:

uv pip install patchelf
CUDA_RT=".venv/lib/python3.12/site-packages/nvidia/cuda_runtime/lib"
find .venv -name "*.so" -exec sh -c \
  'ldd "$1" 2>/dev/null | grep -q "libcudart.so.12 => not found" \
  && patchelf --set-rpath "'"$CUDA_RT"'" "$1" \
  && echo "Patched: $1"' _ {} \;

3. Install system deps for FlashInfer JIT

Required by GDN/Mamba models (Qwen3.5 27B+). You'll hit these errors one by one if any are missing:

sudo apt install -y gcc python3-dev cuda-toolkit ninja-build

4. Serve

Small models (0.8B–9B) — defaults work:

vllm serve Qwen/Qwen3.5-4B \
  --dtype bfloat16 \
  --max-model-len 4096 \
  --max-num-batched-tokens 8192

27B+ GDN models — must use --max-num-batched-tokens 2096:

[github-actions]

This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you!

[github-actions]

This issue has been automatically closed due to inactivity. Please feel free to reopen if you feel it is still relevant. Thank you!