Release list

vLLM 0.28.1rc1 - DeepSeek-V4.1 Flash Vision9 / GLM-5.3 SM89+SM120 (cu130/py3.12) Latest

Latest

yhfgyyf released this 12 Sep 02:18

v0.28.1rc1-vision9-sm89-sm120-cu130

c897ad5

vLLM SM89 + SM120 Vision9 Wheels

DeepSeek-V4.1-Flash support:

Added the native DeepSeek-V4.1 model path, Engram, DSpark, and experimental
CED prefill while retaining the existing DeepSeek-V4 / Vision and
GLM-5.3-Flash support.
Validated the full DeepSeek-V4.1-Flash server on 4× RTX PRO 6000 (SM120).
SM89 validation was limited to offline compilation; no kernel numerical
execution or full-model validation was performed on SM89 hardware.
For the tested long-text prefill workloads, the prefill proxy (input tokens /
TTFT) roughly doubled relative to CED off. The gain is workload-dependent.

The vLLM release wheel reuses the audited, unchanged vision8 native
extensions. The V4.1 additions are Python and FlashInfer JIT source changes.
Use the source-build instructions below when a fresh C++ / CUDA build is
required.

Environment

Linux x86_64
Python 3.12
CUDA Toolkit 13.0
PyTorch 2.13.0+cu130
Triton 3.7.1 with the CUDA 13.1 ptxas-blackwell toolchain
Transformers 5.16.1
FlashInfer 0.6.18+glm53.dsv41.vision2.sm89sm120.cu130.pt213
vLLM 0.28.1rc1.dev517+glm53.dsv41.vision9.sm89sm120.cu130

Installation

uv venv --python 3.12 --seed
source .venv/bin/activate

gh release download v0.28.1rc1-vision9-sm89-sm120-cu130 \
  --repo yhfgyyf/vllm-deepseek-v4-sm89 \
  --pattern 'flashinfer_python-0.6.18+glm53.dsv41.vision2.sm89sm120.cu130.pt213-*.whl' \
  --pattern 'vllm-*glm53.dsv41.vision9.sm89sm120.cu130-*.whl' \
  --pattern SHA256SUMS \
  --dir /tmp/vllm-sm89-sm120-vision9-release

cd /tmp/vllm-sm89-sm120-vision9-release
sha256sum -c SHA256SUMS

UV_DEFAULT_INDEX=https://mirrors.aliyun.com/pypi/simple \
uv pip install ./vllm-*glm53.dsv41.vision9.sm89sm120.cu130-*.whl \
  --torch-backend=cu130
uv pip install 'transformers==5.16.1' 'triton==3.7.1'

The vLLM wheel installs the paired FlashInfer wheel from a pinned URL in this
release. The existing vision7 Docker image does not include
DeepSeek-V4.1-Flash.

Full source build

Prerequisites are the CUDA 13.0 toolkit, a C++ compiler, and Rust/Cargo with
Rust 2024 edition support.

git clone --branch main \
  https://github.com/yhfgyyf/vllm-deepseek-v4-sm89.git
cd vllm-deepseek-v4-sm89

uv venv --python 3.12 --seed
source .venv/bin/activate
uv pip install -r requirements/build/cuda.txt --torch-backend=cu130
uv pip install -r requirements/cuda.txt --torch-backend=cu130
uv pip install 'transformers==5.16.1' 'triton==3.7.1'

export CUDA_HOME=/usr/local/cuda-13.0
export TORCH_CUDA_ARCH_LIST='8.9;12.0'
export MAX_JOBS=4

VLLM_VERSION_OVERRIDE='0.28.1rc1.dev517+glm53.dsv41.vision9.sm89sm120.cu130' \
  uv build --wheel --no-build-isolation
uv pip install --no-build-isolation \
  dist/vllm-0.28.1rc1.dev517+glm53.dsv41.vision9.sm89sm120.cu130-*.whl

requirements/cuda.txt installs the paired release wheel; its source is the
deepseek-v4.1-native-sm89-sm120 branch in
yhfgyyf/flashinfer, at
commit d4eef3c4.

DeepSeek-V4.1-Flash on SM120

The following text-serving configuration was validated on 4× RTX PRO 6000
96 GB. Engram CPU offload also requires sufficient host memory.
V4.1 uses Model Runner V2; switching to the legacy V1 runner is unsupported:

export CUDA_HOME=/usr/local/cuda-13.0
export FLASHINFER_CUDA_ARCH_LIST=12.0
export VLLM_USE_V2_MODEL_RUNNER=1
export TRITON_PTXAS_BLACKWELL_PATH="$(
  "$VIRTUAL_ENV/bin/python" -c \
    'from pathlib import Path; import triton; print(Path(triton.__file__).parent / "backends/nvidia/bin/ptxas-blackwell")'
)"
"$TRITON_PTXAS_BLACKWELL_PATH" --version

vllm serve /path/to/DeepSeek-V4.1-Flash \
  --served-model-name deepseek-v4.1-flash \
  --host 127.0.0.1 \
  --port 8000 \
  --trust-remote-code \
  --tensor-parallel-size 4 \
  --distributed-executor-backend mp \
  --enable-expert-parallel \
  --moe-backend auto \
  --kv-cache-dtype fp8 \
  --block-size 128 \
  --max-model-len auto \
  --max-num-seqs 20 \
  --max-num-batched-tokens 8192 \
  --gpu-memory-utilization 0.90 \
  --enable-prefix-caching \
  --engram-config '{"cpu_offload":true}' \
  --load-format safetensors \
  --safetensors-load-strategy lazy \
  --tokenizer-mode deepseek_v41 \
  --reasoning-parser deepseek_v41 \
  --enable-auto-tool-choice \
  --tool-call-parser deepseek_v41 \
  --hf-overrides '{"ced_prefill":true}' \
  --speculative-config \
  '{"method":"dspark","num_speculative_tokens":5,"draft_sample_method":"probabilistic","rejection_sample_method":"block","enable_adaptive_verification":false}'

The ptxas-blackwell version check must report the CUDA 13.1 toolchain.

CED is an experimental approximate text-prefill path and is not guaranteed to
be output-equivalent to the non-CED path. Validate quality for the target
workload. It does not support multimodal inputs, prompt embeddings, or prompt
logprobs; do not send images or other multimodal content while CED is enabled.

Assets 5

vLLM 0.28.1rc1 - DeepSeek-V4 Vision8 / GLM-5.3 SM89+SM120 (cu130/py3.12)

yhfgyyf released this 07 Sep 06:42

v0.28.1rc1-vision8-sm89-sm120-cu130

0d01ff3

vLLM SM89 + SM120 Vision8 Wheels

Bug fixes:

Fixed the DeepSeek-V4 C128 sparse-attention crash caused by non-contiguous indices reported in Issue #98 (PR #96).
Fixed persistent Top-K candidate-buffer overflow and incorrect index selection (PR #97).
Fixed E8M0 scale compatibility in CUDA Triton block-FP8.

Environment

Linux x86_64
Python 3.12
CUDA Toolkit 13.0
PyTorch 2.13.0+cu130
Triton 3.7.1
FlashInfer 0.6.18+glm53.dsv4.vision1.sm89sm120.cu130.pt213
vLLM 0.28.1rc1.dev517+glm53.dsv4.vision8.sm89sm120.cu130

Installation

uv venv --python 3.12 --seed
source .venv/bin/activate

gh release download v0.28.1rc1-vision8-sm89-sm120-cu130 \
  --repo yhfgyyf/vllm-deepseek-v4-sm89 \
  --pattern 'flashinfer_python-0.6.18+glm53.dsv4.vision1.sm89sm120.cu130.pt213-*.whl' \
  --pattern 'vllm-*glm53.dsv4.vision*.sm89sm120.cu130-*.whl' \
  --pattern SHA256SUMS \
  --dir /tmp/vllm-sm89-sm120-vision8-release

cd /tmp/vllm-sm89-sm120-vision8-release
sha256sum -c SHA256SUMS

UV_DEFAULT_INDEX=https://mirrors.aliyun.com/pypi/simple \
uv pip install ./vllm-*glm53.dsv4.vision*.sm89sm120.cu130-*.whl \
  --torch-backend=cu130

DeepSeek-V4-Flash on SM89 (4x RTX 4090 48 GB)

vllm serve /path/to/DeepSeek-V4-Flash-0731 \
  --served-model-name deepseek-ai/DeepSeek-V4-Flash-0731 \
  --trust-remote-code \
  --tensor-parallel-size 4 \
  --enable-expert-parallel \
  --moe-backend auto \
  --attention-backend FLASHINFER_MLA_SPARSE_DSV4 \
  --kv-cache-dtype fp8_ds_mla \
  --block-size 256 \
  --max-model-len auto \
  --max-num-seqs 4 \
  --max-num-batched-tokens 2048 \
  --gpu-memory-utilization 0.986 \
  --cudagraph-capture-sizes 1 2 4 7 8 \
  --enable-prefix-caching \
  --tokenizer-mode deepseek_v4 \
  --reasoning-parser deepseek_v4 \
  --enable-auto-tool-choice \
  --tool-call-parser deepseek_v4 \
  --speculative-config \
  '{"method":"dspark","num_speculative_tokens":7,"draft_sample_method":"probabilistic"}' \
  --port 8000

DeepSeek-V4-Flash on SM120 (4x RTX PRO 6000 Blackwell 96 GB)

vllm serve /path/to/DeepSeek-V4-Flash-0731 \
  --served-model-name deepseek-ai/DeepSeek-V4-Flash-0731 \
  --trust-remote-code \
  --tensor-parallel-size 4 \
  --enable-expert-parallel \
  --moe-backend auto \
  --attention-backend FLASHINFER_MLA_SPARSE_DSV4 \
  --kv-cache-dtype fp8_ds_mla \
  --block-size 256 \
  --max-model-len auto \
  --max-num-seqs 4 \
  --max-num-batched-tokens 8192 \
  --gpu-memory-utilization 0.95 \
  --enable-prefix-caching \
  --tokenizer-mode deepseek_v4 \
  --reasoning-parser deepseek_v4 \
  --enable-auto-tool-choice \
  --tool-call-parser deepseek_v4 \
  --speculative-config \
  '{"method":"dspark","num_speculative_tokens":7,"draft_sample_method":"probabilistic"}' \
  --port 8000

DeepSeek-V4-Flash-Vision-Exp on SM120 (4x RTX PRO 6000 Blackwell 96 GB)

vllm serve /path/to/DeepSeek-V4-Flash-Vision-Exp \
  --served-model-name deepseek-ai/DeepSeek-V4-Flash-Vision-Exp \
  --tensor-parallel-size 4 \
  --enable-expert-parallel \
  --moe-backend auto \
  --attention-backend FLASHINFER_MLA_SPARSE_DSV4 \
  --kv-cache-dtype fp8_ds_mla \
  --block-size 256 \
  --max-model-len auto \
  --max-num-seqs 4 \
  --max-num-batched-tokens 8192 \
  --gpu-memory-utilization 0.95 \
  --enable-prefix-caching \
  --interleave-mm-strings \
  --tokenizer-mode deepseek_v4 \
  --reasoning-parser deepseek_v4 \
  --enable-auto-tool-choice \
  --tool-call-parser deepseek_v4 \
  --speculative-config \
  '{"method":"dspark","num_speculative_tokens":3}' \
  --port 8000

GLM-5.3-Flash on SM120 (4x RTX PRO 6000 Blackwell 96 GB)

vllm serve /path/to/GLM-5.3-Flash \
  --served-model-name zai-org/GLM-5.3-Flash \
  --tensor-parallel-size 4 \
  --attention-backend FLASHINFER_MLA_SPARSE_SM120 \
  --kv-cache-dtype fp8 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":5}' \
  --reasoning-parser glm45 \
  --enable-auto-tool-choice \
  --tool-call-parser glm47 \
  --block-size 2304 \
  --max-model-len auto \
  --max-num-seqs 4 \
  --max-num-batched-tokens 8192 \
  --gpu-memory-utilization 0.97 \
  --enable-prefix-caching \
  --port 8000

Assets 5

vLLM 0.28.1rc0 - DeepSeek-V4 Vision7 / GLM-5.3 SM89+SM120 (cu130/py3.12)

yhfgyyf released this 01 Sep 07:28

v0.28.1rc0-vision-sm89-sm120-cu130

cb7a435

vLLM SM89 + SM120 Vision7 Wheels

This build includes the Issue #90 fix and replaces the affected vision6 vLLM wheel.

Environment

Linux x86_64
Python 3.12
CUDA Toolkit 13.0
PyTorch 2.13.0+cu130
Triton 3.7.1
FlashInfer 0.6.18+glm53.dsv4.vision1.sm89sm120.cu130.pt213
vLLM 0.28.1rc0.dev293+gcb7a435391.glm53.dsv4.vision7.sm89sm120.cu130

Installation

uv venv --python 3.12 --seed
source .venv/bin/activate

gh release download v0.28.1rc0-vision-sm89-sm120-cu130 \
  --repo yhfgyyf/vllm-deepseek-v4-sm89 \
  --pattern 'flashinfer_python-0.6.18+glm53.dsv4.vision1.sm89sm120.cu130.pt213-*.whl' \
  --pattern 'vllm-*glm53.dsv4.vision*.sm89sm120.cu130-*.whl' \
  --pattern SHA256SUMS \
  --dir /tmp/vllm-sm89-sm120-vision-release

cd /tmp/vllm-sm89-sm120-vision-release
sha256sum -c SHA256SUMS

UV_DEFAULT_INDEX=https://mirrors.aliyun.com/pypi/simple \
uv pip install ./vllm-*glm53.dsv4.vision*.sm89sm120.cu130-*.whl \
  --torch-backend=cu130

DeepSeek-V4-Flash on SM89 (4x RTX 4090 48 GB)

vllm serve /path/to/DeepSeek-V4-Flash-0731 \
  --served-model-name deepseek-ai/DeepSeek-V4-Flash-0731 \
  --trust-remote-code \
  --tensor-parallel-size 4 \
  --enable-expert-parallel \
  --moe-backend auto \
  --attention-backend FLASHINFER_MLA_SPARSE_DSV4 \
  --kv-cache-dtype fp8_ds_mla \
  --block-size 256 \
  --max-model-len auto \
  --max-num-seqs 4 \
  --max-num-batched-tokens 2048 \
  --gpu-memory-utilization 0.986 \
  --cudagraph-capture-sizes 1 2 4 7 8 \
  --enable-prefix-caching \
  --tokenizer-mode deepseek_v4 \
  --reasoning-parser deepseek_v4 \
  --enable-auto-tool-choice \
  --tool-call-parser deepseek_v4 \
  --speculative-config \
  '{"method":"dspark","num_speculative_tokens":7,"draft_sample_method":"probabilistic"}' \
  --port 8000

DeepSeek-V4-Flash on SM120 (4x RTX PRO 6000 Blackwell 96 GB)

vllm serve /path/to/DeepSeek-V4-Flash-0731 \
  --served-model-name deepseek-ai/DeepSeek-V4-Flash-0731 \
  --trust-remote-code \
  --tensor-parallel-size 4 \
  --enable-expert-parallel \
  --moe-backend auto \
  --attention-backend FLASHINFER_MLA_SPARSE_DSV4 \
  --kv-cache-dtype fp8_ds_mla \
  --block-size 256 \
  --max-model-len auto \
  --max-num-seqs 4 \
  --max-num-batched-tokens 8192 \
  --gpu-memory-utilization 0.95 \
  --enable-prefix-caching \
  --tokenizer-mode deepseek_v4 \
  --reasoning-parser deepseek_v4 \
  --enable-auto-tool-choice \
  --tool-call-parser deepseek_v4 \
  --speculative-config \
  '{"method":"dspark","num_speculative_tokens":7,"draft_sample_method":"probabilistic"}' \
  --port 8000

DeepSeek-V4-Flash-Vision-Exp on SM120 (4x RTX PRO 6000 Blackwell 96 GB)

vllm serve /path/to/DeepSeek-V4-Flash-Vision-Exp \
  --served-model-name deepseek-ai/DeepSeek-V4-Flash-Vision-Exp \
  --tensor-parallel-size 4 \
  --enable-expert-parallel \
  --moe-backend auto \
  --attention-backend FLASHINFER_MLA_SPARSE_DSV4 \
  --kv-cache-dtype fp8_ds_mla \
  --block-size 256 \
  --max-model-len auto \
  --max-num-seqs 4 \
  --max-num-batched-tokens 8192 \
  --gpu-memory-utilization 0.95 \
  --enable-prefix-caching \
  --interleave-mm-strings \
  --tokenizer-mode deepseek_v4 \
  --reasoning-parser deepseek_v4 \
  --enable-auto-tool-choice \
  --tool-call-parser deepseek_v4 \
  --speculative-config \
  '{"method":"dspark","num_speculative_tokens":3}' \
  --port 8000

GLM-5.3-Flash on SM120 (4x RTX PRO 6000 Blackwell 96 GB)

vllm serve /path/to/GLM-5.3-Flash \
  --served-model-name zai-org/GLM-5.3-Flash \
  --tensor-parallel-size 4 \
  --attention-backend FLASHINFER_MLA_SPARSE_SM120 \
  --kv-cache-dtype fp8 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":5}' \
  --reasoning-parser glm45 \
  --enable-auto-tool-choice \
  --tool-call-parser glm47 \
  --block-size 2304 \
  --max-model-len auto \
  --max-num-seqs 4 \
  --max-num-batched-tokens 8192 \
  --gpu-memory-utilization 0.97 \
  --enable-prefix-caching \
  --port 8000

Assets 5

vLLM 0.28.1rc0 - DeepSeek-V4 / GLM-5.3 SM89+SM120 (cu130/py3.12)

yhfgyyf released this 30 Aug 13:26

v0.28.1rc0-sm89-sm120-cu130

5911abc

vLLM SM89 + SM120 Wheels

Environment

Linux x86_64
Python 3.12
CUDA Toolkit 13.0
PyTorch 2.13.0+cu130
Triton 3.7.1
FlashInfer 0.6.18
vLLM 0.28.1rc0-110

Installation

uv venv --python 3.12 --seed
source .venv/bin/activate

gh release download v0.28.1rc0-sm89-sm120-cu130 \
  --repo yhfgyyf/vllm-deepseek-v4-sm89 \
  --pattern 'flashinfer_python-0.6.18+glm53.dsv4.sm89sm120.cu130.pt213-*.whl' \
  --pattern 'vllm-*sm89sm120.cu130-*.whl' \
  --pattern SHA256SUMS \
  --dir /tmp/vllm-sm89-sm120-release

cd /tmp/vllm-sm89-sm120-release
sha256sum -c SHA256SUMS

UV_DEFAULT_INDEX=https://mirrors.aliyun.com/pypi/simple \
uv pip install ./vllm-*.whl --torch-backend=cu130

DeepSeek-V4-Flash on SM89 (4x RTX 4090 48 GB)

vllm serve /path/to/DeepSeek-V4-Flash-0731 \
  --served-model-name deepseek-ai/DeepSeek-V4-Flash-0731 \
  --trust-remote-code \
  --tensor-parallel-size 4 \
  --enable-expert-parallel \
  --moe-backend auto \
  --attention-backend FLASHINFER_MLA_SPARSE_DSV4 \
  --kv-cache-dtype fp8_ds_mla \
  --block-size 256 \
  --max-model-len 131584 \
  --max-num-seqs 4 \
  --max-num-batched-tokens 2048 \
  --gpu-memory-utilization 0.986 \
  --cudagraph-capture-sizes 1 2 4 7 8 \
  --enable-prefix-caching \
  --tokenizer-mode deepseek_v4 \
  --reasoning-parser deepseek_v4 \
  --enable-auto-tool-choice \
  --tool-call-parser deepseek_v4 \
  --speculative-config \
  '{"method":"dspark","num_speculative_tokens":7,"draft_sample_method":"probabilistic"}' \
  --port 8000

DeepSeek-V4-Flash on SM120 (4x RTX PRO 6000 Blackwell 96 GB)

vllm serve /path/to/DeepSeek-V4-Flash-0731 \
  --served-model-name deepseek-ai/DeepSeek-V4-Flash-0731 \
  --trust-remote-code \
  --tensor-parallel-size 4 \
  --enable-expert-parallel \
  --moe-backend auto \
  --attention-backend FLASHINFER_MLA_SPARSE_DSV4 \
  --kv-cache-dtype fp8_ds_mla \
  --block-size 256 \
  --max-model-len auto \
  --max-num-seqs 4 \
  --max-num-batched-tokens 8192 \
  --gpu-memory-utilization 0.95 \
  --enable-prefix-caching \
  --tokenizer-mode deepseek_v4 \
  --reasoning-parser deepseek_v4 \
  --enable-auto-tool-choice \
  --tool-call-parser deepseek_v4 \
  --speculative-config \
  '{"method":"dspark","num_speculative_tokens":7,"draft_sample_method":"probabilistic"}' \
  --port 8000

GLM-5.3-Flash on SM120 (4x RTX PRO 6000 Blackwell 96 GB)

vllm serve /path/to/GLM-5.3-Flash \
  --served-model-name zai-org/GLM-5.3-Flash \
  --tensor-parallel-size 4 \
  --attention-backend FLASHINFER_MLA_SPARSE_SM120 \
  --kv-cache-dtype fp8 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":5}' \
  --reasoning-parser glm45 \
  --enable-auto-tool-choice \
  --tool-call-parser glm47 \
  --block-size 2304 \
  --max-model-len auto \
  --max-num-seqs 4 \
  --max-num-batched-tokens 8192 \
  --gpu-memory-utilization 0.97 \
  --enable-prefix-caching \
  --port 8000

Assets 5

vLLM 0.23.1rc1.dev904 - DeepSeek-V4 SM89 Upstream Sync (cu132/py3.12)

yhfgyyf released this 22 Aug 07:49

v0.23.1rc1.dev904-g998fd644b-cu132-sm89

998fd64

vLLM DeepSeek-V4 SM89 CUDA 13.2 Release

中文

用途

这是面向 SM89/Ada 的 DeepSeek-V4-Flash-0731 更新版，选择性同步 vLLM v0.27.1 时期的 DeepSeek-V4 / DSpark 修复与优化，并升级到 CUDA toolkit 13.2、torch 2.13/cu130 和 FlashInfer 0.6.17 SM89 sparse MLA fork。

主要更新

同步 tokenizer/parser、DSpark backend 继承、DFlash hybrid causal metadata、mHC broadcast、DSV4 top-k、sparse index metadata 和 sparse MLA/SWA correctness 更新。
保留 SM89 FlashInfer sparse MLA 的宽 eager CUDA Graph guard，避免 indexer 输出顺序问题；未启用 confidence-scheduled adaptive verification。
包含本 fork 的 SM89 paged MQA logits int32 地址溢出修复（PR #51）和 Triton per-shape kernel cache 增长修复（PR #61）。
FlashInfer Python 0.6.17+sm89.2 修复 TP8 下 DSV4 sparse MLA prefill num_heads=8 的问题。
DSpark 推荐参数：{"method":"dspark","num_speculative_tokens":7,"draft_sample_method":"probabilistic"}。

构建环境

Python 3.12.13
CUDA toolkit 13.2；torch 2.13.0+cu130；Triton 3.7.1
FlashInfer Python 0.6.17+sm89.2；FlashInfer cubin 0.6.17
TORCH_CUDA_ARCH_LIST=8.9+PTX，面向 RTX 4090 / L40 / L40S / L4 / RTX 6000 Ada

当前关键资产

资产	SHA256
`flashinfer_python-0.6.17+sm89.2-py3-none-any.whl`	`ab255bf648d7d00498284ad05aa64265cbd0fefd1c9e95265534f6b1e840fe00`
`vllm-0.23.1rc1.dev904+sm89.cu132-cp312-cp312-linux_x86_64.whl`	`ad9d78b4a2852a16491f4dc3b7d90703df182309b8709c51ad1fe173fbfc3c53`

安装

uv venv --python 3.12 --seed
source .venv/bin/activate

gh release download v0.23.1rc1.dev904-g998fd644b-cu132-sm89 \
  --repo yhfgyyf/vllm-deepseek-v4-sm89 \
  --pattern '*.whl' \
  --dir /tmp/vllm-sm89-release

UV_DEFAULT_INDEX=https://mirrors.aliyun.com/pypi/simple \
uv pip install /tmp/vllm-sm89-release/*.whl \
  --torch-backend=cu130

export FLASHINFER_DISABLE_VERSION_CHECK=1

验证

4× RTX 4090、DeepSeek-V4-Flash-0731、DSpark k=7 probabilistic。
单并发 8K / 32K -> 1K：prefill 4891.19 / 4859.82 tok/s，decode 366.95 / 327.38 tok/s，接受率 95.63% / 83.05%。
4 并发、输出 512 的源码/wheel A/B：8K 输出吞吐 210.74 / 216.64 tok/s；32K 为 70.98 / 70.45 tok/s；均 4/4 成功。
32K 单并发、输出 512：端到端 output throughput 61.82 tok/s；TPOT 3.047 ms，对应纯 decode 328.2 tok/s；接受率 84.08%。
新 vLLM wheel 已通过压缩包完整性、RECORD、依赖元数据和逐文件检查；除 METADATA 与重新生成的 RECORD 外，其余 4471 个文件与原 wheel 逐字节一致。

资产完整性请用本 Release 的 SHA256SUMS 和 MANIFEST.json 校验。vLLM wheel、FlashInfer Python wheel 与 FlashInfer cubin wheel 必须使用同一 Release 中的组合。

English

Purpose

This release refreshes DeepSeek-V4-Flash-0731 for SM89/Ada with selected DeepSeek-V4 / DSpark fixes and optimizations from the vLLM v0.27.1 era. It upgrades the release stack to CUDA toolkit 13.2, torch 2.13/cu130, and the FlashInfer 0.6.17 SM89 sparse MLA fork.

Highlights

Synced tokenizer/parser fixes, DSpark backend inheritance, DFlash hybrid causal metadata, mHC broadcast, DSV4 top-k, sparse-index metadata, and sparse MLA/SWA correctness updates.
Retained the SM89 FlashInfer sparse MLA wide-eager CUDA Graph guard to avoid indexer output-ordering failures. Confidence-scheduled adaptive verification is not enabled.
Includes this fork's SM89 paged-MQA-logits int32 addressing fix (PR #51) and Triton per-shape kernel-cache growth fix (PR #61).
FlashInfer Python 0.6.17+sm89.2 fixes DSV4 sparse MLA prefill with num_heads=8 under TP8.
Recommended DSpark config: {"method":"dspark","num_speculative_tokens":7,"draft_sample_method":"probabilistic"}.

Build Environment

Python 3.12.13
CUDA toolkit 13.2; torch 2.13.0+cu130; Triton 3.7.1
FlashInfer Python 0.6.17+sm89.2; FlashInfer cubin 0.6.17
TORCH_CUDA_ARCH_LIST=8.9+PTX, targeting RTX 4090 / L40 / L40S / L4 / RTX 6000 Ada

Current Key Assets

Asset	SHA256
`flashinfer_python-0.6.17+sm89.2-py3-none-any.whl`	`ab255bf648d7d00498284ad05aa64265cbd0fefd1c9e95265534f6b1e840fe00`
`vllm-0.23.1rc1.dev904+sm89.cu132-cp312-cp312-linux_x86_64.whl`	`ad9d78b4a2852a16491f4dc3b7d90703df182309b8709c51ad1fe173fbfc3c53`

Install

Use the commands in the Chinese section above, or the equivalent installation instructions in README_EN.md.

Validation

4× RTX 4090 with DeepSeek-V4-Flash-0731 and DSpark k=7 probabilistic.
Single concurrency, 8K / 32K -> 1K: 4891.19 / 4859.82 tok/s prefill, 366.95 / 327.38 tok/s decode, and 95.63% / 83.05% acceptance.
Four-concurrency, 512-output source/wheel A/B: 210.74 / 216.64 output tok/s at 8K and 70.98 / 70.45 at 32K; all runs completed 4/4.
Single-concurrency 32K -> 512: 61.82 end-to-end output tok/s; 3.047 ms TPOT, equivalent to 328.2 decode tok/s; 84.08% acceptance.
The new vLLM wheel passed archive-integrity, RECORD, dependency-metadata, and file-by-file checks. All 4,471 files other than METADATA and the regenerated RECORD are byte-identical to the original wheel.

Verify assets with the SHA256SUMS and MANIFEST.json files attached to this Release. Install the vLLM, FlashInfer Python, and FlashInfer cubin wheels from this same Release as one matched set.

Assets 7

vLLM 0.23.1rc1.dev904 - DeepSeek-V4-Flash-0731 SM89 Accuracy Fix (cu130/py3.12)

yhfgyyf released this 02 Aug 09:10

v0.23.1rc1.dev904-g8e321cc4f-cu130-sm89

8e321cc

Purpose

CUDA 13.0 / torch cu130 wheels for the latest DeepSeek-V4-Flash-0731 model
on NVIDIA Ada SM89 GPUs.

This release fixes the FlashInfer SM89 sparse MLA decode accuracy path. The
software FP8 MMA fallback now reconstructs the distributed query and KV UE8M0
scales for each accumulator, handles the UE8M0 zero and NaN encodings, and
supports the DeepSeek V4 page block size of 256.

The vLLM wheel also converts an unavailable optional DeepGEMM import into a
concise warning instead of printing a startup traceback. It requires the
validated flashinfer-python==0.6.14+sm89.1 build, so the unpatched official
package cannot satisfy the runtime dependency accidentally.

Build Environment

Python: 3.12
vLLM wheel version: 0.23.1rc1.dev904+g8e321cc4f.cu130
vLLM build commit: 8e321cc
Repository main documentation commit: 47851d9
FlashInfer wheel version: 0.6.14+sm89.1
FlashInfer source commit: d14d27eb7fcc17afb1fb792b3c2bb9cb132d1354
CUDA toolkit: /usr/local/cuda-13.0, nvcc 13.0.48
PyTorch: 2.11.0+cu130, torch.version.cuda 13.0
Target CUDA codegen: compute_89 / sm_89, with PTX
Platform: Linux x86_64, CPython 3.12 cp312

Assets

vLLM: vllm-0.23.1rc1.dev904+g8e321cc4f.cu130-cp312-cp312-linux_x86_64.whl
- SHA256: d1a4e4ee3f64882f129a8d12e47bcd70c46992b83a16c2e6c722d85dbe87b41c
FlashInfer: flashinfer_python-0.6.14+sm89.1-py3-none-any.whl
- SHA256: 667e4c1c1a288681493e192a0d02976e791078a8792c47bad4d7f9186109554e

Install

Install both release assets in one resolver invocation so the CuTe DSL
dependency remains on the vLLM-validated 4.5.2 build.

uv venv --python 3.12 --seed
source .venv/bin/activate

uv pip install \
  "https://github.com/yhfgyyf/vllm-deepseek-v4-sm89/releases/download/v0.23.1rc1.dev904-g8e321cc4f-cu130-sm89/flashinfer_python-0.6.14%2Bsm89.1-py3-none-any.whl" \
  "https://github.com/yhfgyyf/vllm-deepseek-v4-sm89/releases/download/v0.23.1rc1.dev904-g8e321cc4f-cu130-sm89/vllm-0.23.1rc1.dev904%2Bg8e321cc4f.cu130-cp312-cp312-linux_x86_64.whl" \
  "flashinfer-cubin==0.6.13" \
  --torch-backend=cu130

export FLASHINFER_DISABLE_VERSION_CHECK=1
export LD_LIBRARY_PATH="$VIRTUAL_ENV/lib/python3.12/site-packages/PyNvVideoCodec:$VIRTUAL_ENV/lib:${LD_LIBRARY_PATH:-}"

Use --attention-backend FLASHINFER_MLA_SPARSE_DSV4 for the documented
DeepSeek-V4-Flash deployment.

Notes

This release targets CUDA 13.0 / cu130 and exact compute capability 8.9. It
is not intended for CUDA 12, cu128, or non-SM89 runtime deployment.
The FlashInfer asset comes from
yhfgyyf/flashinfer@d14d27eb.
Official FlashInfer 0.6.14 does not include this SM89 sparse MLA accuracy
fix and does not satisfy the wheel's exact local-version pin.
flashinfer-cubin==0.6.13 is intentional because matching 0.6.14 cubins are
not published on the configured index. SM89 sparse MLA is JIT-compiled from
the patched 0.6.14 Python package.
The PyNvVideoCodec library path is only needed when the host does not
provide a complete supported FFmpeg shared-library installation.
DeepGEMM does not need to be uninstalled. On SM89 its unavailable extension
produces one concise optional-module warning per worker and vLLM continues
with the supported fallback paths.
This release does not update confidence_head and does not include
per-request ragged verify. The validated DSpark configuration used six
speculative tokens and CUDA Graphs remained enabled.
The release tag points to the exact vLLM wheel build commit. Repository
main contains that commit plus the bilingual changelog update.

Validation

The final FlashInfer wheel passed all seven targeted sparse MLA decode tests
on a real RTX 4090 (SM89), including page block sizes 64 and 256, distributed
scale reconstruction, and the minimum UE8M0 scale boundary.
The final vLLM wheel passed seven optional-import regression tests. All
changed files also passed the repository pre-commit hooks.
A fresh remote Python environment installed the two exact wheel assets.
uv pip check reported all 199 packages compatible after resolving CuTe DSL
to 4.5.2.
Installed vLLM and FlashInfer source hashes matched the local source and
packaged wheel contents. Both assets passed ZIP, METADATA, and SHA256 checks.
Full model accuracy validation completed on four RTX 4090 GPUs: short
contexts passed 8/8, a 119,996-token needle retrieval returned the exact
target, and the ModelScope GSM8K run completed 1,319/1,319 samples with
1,188 exact matches (90.068%) and no request failures.
Exact-wheel smoke tests returned 391 for 17 * 23 and recovered
SM89-0731-3917 from a 9,943-token chat prompt without output corruption.
The final 8,192-input / 256-output / concurrency-1 serving benchmark
completed 1/1 requests with no failures: TTFT 2054.41 ms, prefill 3987.52
tok/s, decode-only 250.67 tok/s, and DSpark acceptance rate 68.67%.
Runtime logs confirmed sparse MLA warmup on all four workers, the SM89 direct
FP8 MQA top-k path, full and piecewise CUDA Graph capture, DSpark graph
capture, and HTTP health status 200. The final log contained zero errors,
zero tracebacks, and no CuTe DSL import failure.
cuobjdump found 38 SM89 cubins in the main stable extension and 12 SM89
cubins in the MoE stable extension. Unsupported DeepGEMM and FlashMLA
extensions are not packaged.

Assets 4

vLLM 0.23.1rc1.dev904 - DeepSeek-V4 DSpark SM89 Issue #39 Fix (cu130/py3.12)

yhfgyyf released this 24 Jul 09:28

v0.23.1rc1.dev904-g83467b5e7-cu130-sm89

83467b5

Purpose

CUDA 13.0 / torch cu130 wheels for the DeepSeek-V4-Flash DSpark branch on
NVIDIA Ada SM89 GPUs.

This release fixes Issue #39:
deep_gemm_fp8_o_proj could pass flat TP-sharded weight and block-scale
tensors into fp8_einsum, which then failed its rank check with
t.dim() == N. The DeepSeek V4 wrapper now reconstructs the grouped
[groups, out_rank, hidden] weight and scale layouts, selects the TP-local
group slice, and dispatches the SM89 Triton fallback without entering the
unsupported DeepGEMM FP8 einsum path.

The wheel also requires the validated flashinfer-python==0.6.14+sm89
build, so the unpatched official 0.6.14 package can no longer satisfy the
runtime dependency accidentally.

Build Environment

Python: 3.12.13
vLLM wheel version: 0.23.1rc1.dev904+g83467b5e7
vLLM build commit: 83467b5
Repository main merge commit: a3ab48e
FlashInfer wheel version: 0.6.14+sm89
FlashInfer source commit: 1227d9b42281a97bbbb04bbda7cda505a38ae247
CUDA toolkit: /usr/local/cuda-13.0, nvcc 13.0.48
PyTorch: 2.11.0+cu130, torch.version.cuda 13.0
Target CUDA codegen: compute_89 / sm_89
Platform: Linux x86_64, CPython 3.12 cp312

Assets

vLLM: vllm-0.23.1rc1.dev904+g83467b5e7-cp312-cp312-linux_x86_64.whl
- SHA256: 22fbeaa684b1f7555a81aaa203046c1fdff6c98837ab650cf967c462895e2d0c
FlashInfer: flashinfer_python-0.6.14+sm89-py3-none-any.whl
- SHA256: 95ea827b9a6303fc974f7b2872befb23efed9a3eb85074b262261e3c3944730b

Install

uv venv --python 3.12 --seed
source .venv/bin/activate

uv pip install torch==2.11.0 flashinfer-cubin==0.6.13 --torch-backend=cu130
uv pip install \
  "https://github.com/yhfgyyf/vllm-deepseek-v4-sm89/releases/download/v0.23.1rc1.dev904-g83467b5e7-cu130-sm89/flashinfer_python-0.6.14%2Bsm89-py3-none-any.whl"
uv pip install \
  "https://github.com/yhfgyyf/vllm-deepseek-v4-sm89/releases/download/v0.23.1rc1.dev904-g83467b5e7-cu130-sm89/vllm-0.23.1rc1.dev904%2Bg83467b5e7-cp312-cp312-linux_x86_64.whl" \
  --torch-backend=cu130
export FLASHINFER_DISABLE_VERSION_CHECK=1

Use --attention-backend FLASHINFER_MLA_SPARSE_DSV4 for the documented
DeepSeek-V4-Flash deployment.

Notes

This release targets CUDA 13.0 / cu130 and exact compute capability 8.9. It
is not intended for CUDA 12, cu128, or non-SM89 runtime deployment.
The FlashInfer asset comes from
yhfgyyf/flashinfer@1227d9b4.
Official FlashInfer 0.6.14 does not include the required SM89 sparse MLA
patch and does not satisfy this vLLM wheel's exact local-version pin.
flashinfer-cubin==0.6.13 is intentional because matching 0.6.14 cubins are
not published on the configured index. SM89 sparse MLA is JIT-compiled from
the patched 0.6.14 Python package.
DeepGEMM does not need to be uninstalled. Unsupported scheduler metadata and
FP8 einsum paths are selected by hardware capability, not package presence.
This release does not update confidence_head and does not restore
per-request ragged verify. The validated DSpark configuration used
num_speculative_tokens=7.
The release tag points to the exact wheel build commit. The repository
main branch contains that commit through merge commit a3ab48eb6, together
with the existing repository documentation and SM89 capability probes.

Validation

Issue #39 regression coverage includes flat TP-sharded FP8 weight and scale
reconstruction, TP-local group slicing, and the SM89 wrapper fallback.
Remote SM89 targeted tests completed with 16 passed and 8 skipped. The final
main merge added 11 passing integration tests and passed pre-commit on all
merge-specific Python, shell, Markdown, generated documentation, and
dependency files.
CUDA Graph validation completed for PIECEWISE 11/11, FULL 4/4, and DSpark
FULL 4/4 captures.
Runtime logs showed the SM89 direct FP8 MQA top-k path, sparse MLA warmup on
all four workers, DSpark loading, and the custom FlashInfer
0.6.14+sm89/89/.../sparse_mla_sm120.so JIT module.
Two end-to-end inference requests returned HTTP 200. DSpark mean accepted
lengths were 3.27 and 3.10, with average draft acceptance rates of 32.4%
and 29.9%. The Issue #39 rank assertion did not recur.
The final vLLM wheel passed ZIP integrity, METADATA, packaged-source
equality, and SHA256 checks. METADATA pins torch==2.11.0,
flashinfer-python==0.6.14+sm89, flashinfer-cubin==0.6.13, and
nvidia-cutlass-dsl[cu13]==4.5.2.
cuobjdump found 38 SM89 cubins in the main stable extension and 12 SM89
cubins in the MoE stable extension. Unsupported DeepGEMM and FlashMLA
extensions are not packaged.
The remote SM89 runtime validation used the same source code before the
final metadata-only FlashInfer local-version pin. The final wheel passed all
static artifact checks, but the target server was shut down before that
exact wheel could be reinstalled.
The broader CUDA slot-mapping test still cannot complete because its gated
meta-llama/Meta-Llama-3-8B fixture returns HTTP 401; the merge-specific
scheduler metadata tests pass.

Assets 4

vLLM 0.23.1rc1.dev1018 - DeepSeek-V4 DSpark SM89 FlashInfer 0.6.14 (cu130/py3.12)

yhfgyyf released this 10 Jul 07:29

v0.23.1rc1.dev1018-g8aba6ae7e-cu130-sm89

8aba6ae

Purpose

CUDA 13.0 / torch cu130 wheels for the DeepSeek-V4-Flash DSpark branch on NVIDIA Ada SM89 GPUs. This release moves SM89 sparse MLA prefill/decode to the patched FlashInfer 0.6.14 JIT path and gates Lightning Indexer DeepGEMM scheduler metadata by actual hardware support instead of package presence.

Build Environment

Python: 3.12.13
vLLM wheel version: 0.23.1rc1.dev1018+g8aba6ae7e.cu130
vLLM build commit: 8aba6ae
FlashInfer wheel version: 0.6.14+sm89
FlashInfer source commit: 1227d9b42281a97bbbb04bbda7cda505a38ae247
CUDA toolkit: /usr/local/cuda-13.0, nvcc 13.0.48
PyTorch: 2.11.0+cu130, torch.version.cuda 13.0
Target arch: TORCH_CUDA_ARCH_LIST=8.9+PTX
Platform: Linux x86_64, CPython 3.12 cp312

Assets

vLLM: vllm-0.23.1rc1.dev1018+g8aba6ae7e.cu130-cp312-cp312-linux_x86_64.whl
- SHA256: f335d3da9f783b18d6f78f19689e2b5267d75a9eb9bc6f966d16aa7b83e71c92
FlashInfer: flashinfer_python-0.6.14+sm89-py3-none-any.whl
- SHA256: 95ea827b9a6303fc974f7b2872befb23efed9a3eb85074b262261e3c3944730b

Install

uv venv --python 3.12 --seed
source .venv/bin/activate

uv pip install torch==2.11.0 flashinfer-cubin==0.6.13 --torch-backend=cu130
uv pip install \
  "https://github.com/yhfgyyf/vllm-deepseek-v4-sm89/releases/download/v0.23.1rc1.dev1018-g8aba6ae7e-cu130-sm89/flashinfer_python-0.6.14%2Bsm89-py3-none-any.whl"
uv pip install \
  "https://github.com/yhfgyyf/vllm-deepseek-v4-sm89/releases/download/v0.23.1rc1.dev1018-g8aba6ae7e-cu130-sm89/vllm-0.23.1rc1.dev1018%2Bg8aba6ae7e.cu130-cp312-cp312-linux_x86_64.whl" \
  --torch-backend=cu130
export FLASHINFER_DISABLE_VERSION_CHECK=1

Use --attention-backend FLASHINFER_MLA_SPARSE_DSV4 for the documented DeepSeek-V4-Flash deployment.

Notes

This release targets CUDA 13.0 / cu130 and exact compute capability 8.9. It is not intended for CUDA 12, cu128, or non-SM89 runtime deployment.
The FlashInfer asset comes from yhfgyyf/flashinfer@1227d9b4. Official FlashInfer 0.6.14 does not include the required SM89 sparse MLA patch and is rejected by the runtime capability probe.
flashinfer-cubin==0.6.13 is intentional because 0.6.14 cubins are not published on the configured index. SM89 sparse MLA is JIT-compiled from the patched 0.6.14 Python package.
DeepGEMM does not need to be uninstalled. SM89 skips unsupported DeepGEMM scheduler metadata through the hardware capability gate.
This release does not update confidence_head and does not include per-request adaptive ℓ. DSpark remains fixed at ℓ=6.
Source changes were reviewed and merged through vLLM PR #24; the reproducible release-version fix was merged through PR #27.

Validation

Targeted vLLM tests: 14 passed. Pre-commit passed on all affected Python, shell, Markdown, generated documentation, dependency, and Docker consistency files.
CUDA 13.0 build from exact main commit completed with WHEEL_BUILD_OK; CMake emitted sm_89 cubin plus compute_89 PTX flags and skipped unsupported DeepGEMM/QuTLASS compilation on architecture 8.9.
Wheel ZIP integrity, METADATA, packaged-source equality, and SHA256 checks passed. METADATA pins torch==2.11.0, flashinfer-python==0.6.14, and flashinfer-cubin==0.6.13.
A clean uv installation resolved 192 packages with no broken requirements. Imports passed for vLLM, both stable CUDA extensions, FA2, FA3, DSpark, DeepSeek-V4 FlashInfer sparse MLA, and FlashInfer 0.6.14+sm89.
cuobjdump found 38 SM89 cubins and one SM89 PTX image in the main extension, plus 12 SM89 cubins in the MoE extension. ldd resolved all four CUDA/FA extension dependencies against CUDA 13.0 and torch cu130.
The exact wheel was import-validated on a non-target SM120 build host; SM120 kernel execution is not claimed for this SM89-only asset. The supplied SM89 SSH endpoint closed before the SSH banner during final packaging, so exact-wheel reinstall on that host could not be repeated.
The fixed-ℓ=6 implementation was previously validated on 4× RTX 4090, TP=4, single concurrency, with five successful runs per case:

Input -> output	Success	Prefill TPS	Decode TPS
8K -> 1K	5/5	3515.72	286.82
32K -> 1K	5/5	4881.18	344.63
128K -> 1K	5/5	3812.00	313.57

The broader CUDA slot-mapping test could not complete because its gated meta-llama/Meta-Llama-3-8B fixture returned HTTP 401; the newly added test cases in that file passed.

Assets 4

vLLM 0.23.1rc1.dev145 - DeepSeek-V4 DSpark SM80 testing build (cu130/py3.12) Pre-release

Pre-release

yhfgyyf released this 06 Jul 01:15

v0.23.1rc1.dev145-g12d96ee0c-cu130-sm80

12d96ee

Purpose

CUDA 13.0 / torch cu130 wheel for the DeepSeek-V4-Flash DSpark branch on NVIDIA Ampere SM80 GPUs. This is a testing-only SM80 adaptation intended for A100/A800 class systems that need the DeepSeek-only DSpark speculative decoding path.

Build Environment

Python: 3.12.13
vLLM wheel version: 0.23.1rc1.dev145+g12d96ee0c.cu130
Build commit: 12d96ee
CUDA toolkit: /usr/local/cuda-13.0, nvcc 13.0.48
PyTorch: 2.11.0+cu130, torch.version.cuda 13.0
Target arch: TORCH_CUDA_ARCH_LIST=8.0
Platform: Linux x86_64, CPython 3.12 cp312

Asset

Wheel: vllm-0.23.1rc1.dev145+g12d96ee0c.cu130-cp312-cp312-linux_x86_64.whl
SHA256: 658e65895b235e6dd4168ecad9317df71cc22d63ce3fb115911f27323271b72b

Install

uv venv --python 3.12 --seed
source .venv/bin/activate

uv pip install \
  "https://github.com/yhfgyyf/vllm-deepseek-v4-sm89/releases/download/v0.23.1rc1.dev145-g12d96ee0c-cu130-sm80/vllm-0.23.1rc1.dev145%2Bg12d96ee0c.cu130-cp312-cp312-linux_x86_64.whl" \
  --extra-index-url https://download.pytorch.org/whl/cu130

SM80 DSpark Benchmark Snapshot

A800 SM80, max-num-batched-tokens=16384, FlashInfer sampler, sparse MLA warmup, DSpark num_speculative_tokens=6. Decode numbers are per-request TPS from median TPOT, compared with the same mbt16k no-DSpark baseline.

Case	DSpark decode TPS/req	No DSpark decode TPS/req	Ratio
8k -> 1k, c1	229.8	57.6	3.99x
32k -> 1k, c1	274.2	58.1	4.72x

Notes

This SM80 build is experimental and has only been validated on A800-class hardware in the recorded test environment.
This wheel is for CUDA 13.0 / cu130 environments and SM80 Ampere GPUs. It is not intended for cu128 or CUDA 12 deployments.
DSpark support in this branch is scoped to DeepSeek-V4; Qwen3 DSpark is intentionally excluded.

Validation

sm80-deepseek-v4-flash branch pushed to commit 12d96ee.
main documentation update pushed to commit a7374bf.
Wheel METADATA reports Version: 0.23.1rc1.dev145+g12d96ee0c.cu130.
Wheel contains vllm/_C.abi3.so, vllm/_C_stable_libtorch.abi3.so, vllm/_moe_C_stable_libtorch.abi3.so, and DSpark modules.
Import smoke test from a temporary venv passed for vllm, vllm._C, vllm._C_stable_libtorch, vllm._moe_C_stable_libtorch, torch 2.11.0+cu130.
ldd with CUDA 13.0 and torch library paths resolved libcudart.so.13 from /usr/local/cuda-13.0/lib64.
A800 benchmark artifacts were copied locally under bench_compare/dspark_mbt16384_server_data_20260705.

Assets 3

vLLM 0.23.1rc1.dev145 - DeepSeek-V4 DSpark SM89 (cu130/py3.12)

yhfgyyf released this 01 Jul 13:09

v0.23.1rc1.dev145-g8c631d45e-cu130-sm89

8c631d4

Purpose

CUDA 13.0 / torch cu130 wheel for the DeepSeek-V4-Flash DSpark branch on NVIDIA Ada SM89 GPUs. Intended for RTX 4090, L40, L40S, L4, and RTX 6000 Ada class systems that need the DeepSeek-only DSpark speculative decoding path and the documented tool-call parser setup.

Build Environment

Python: 3.12.13
vLLM wheel version: 0.23.1rc1.dev145+g8c631d45e.cu130
Build commit: 8c631d4
CUDA toolkit: /usr/local/cuda-13.0, nvcc 13.0.48
PyTorch: 2.11.0+cu130, torch.version.cuda 13.0
Target arch: TORCH_CUDA_ARCH_LIST=8.9+PTX
Platform: Linux x86_64, CPython 3.12 cp312

Asset

Wheel: vllm-0.23.1rc1.dev145+g8c631d45e.cu130-cp312-cp312-linux_x86_64.whl
SHA256: 80cf656f261f5cf78513c9970bdeef96e9ac2723329141567bfae099dcca54c6

Install

uv venv --python 3.12 --seed
source .venv/bin/activate

pip install \
  "https://github.com/yhfgyyf/vllm-deepseek-v4-sm89/releases/download/v0.23.1rc1.dev145-g8c631d45e-cu130-sm89/vllm-0.23.1rc1.dev145%2Bg8c631d45e.cu130-cp312-cp312-linux_x86_64.whl" \
  --extra-index-url https://download.pytorch.org/whl/cu130

Notes

This wheel is for CUDA 13.0 / cu130 environments and SM89 Ada GPUs. It is not intended for cu128 or CUDA 12 deployments. DSpark support in this branch is scoped to DeepSeek-V4; Qwen3 DSpark is intentionally excluded.

Validation

git diff --cached --check passed before commit
Staged Python files passed .venv/bin/python -m py_compile before commit
CUDA 13.0 build completed with WHEEL_BUILD_OK
Wheel METADATA reports Version: 0.23.1rc1.dev145+g8c631d45e.cu130
Wheel contains vllm/_C.abi3.so, vllm/vllm_flash_attn/_vllm_fa2_C.abi3.so, and DSpark modules
Import smoke test from /tmp passed for vllm, vllm._C, FA2 extension, DSpark model module, DSpark speculator, torch 2.11.0+cu130
ldd with CUDA 13.0 and torch library paths resolved libcudart.so.13 from /usr/local/cuda-13.0/lib64
Direct wheel URL returns the public GitHub release asset, and torch==2.11.0 resolves to torch==2.11.0+cu130 through https://download.pytorch.org/whl/cu130

Assets 3

Releases: yhfgyyf/vllm-deepseek-v4-sm89

Release list

vLLM 0.28.1rc1 - DeepSeek-V4.1 Flash Vision9 / GLM-5.3 SM89+SM120 (cu130/py3.12)

vLLM SM89 + SM120 Vision9 Wheels

Environment

Installation

Full source build

DeepSeek-V4.1-Flash on SM120

Uh oh!

vLLM 0.28.1rc1 - DeepSeek-V4 Vision8 / GLM-5.3 SM89+SM120 (cu130/py3.12)

vLLM SM89 + SM120 Vision8 Wheels

Environment

Installation

DeepSeek-V4-Flash on SM89 (4x RTX 4090 48 GB)

DeepSeek-V4-Flash on SM120 (4x RTX PRO 6000 Blackwell 96 GB)

DeepSeek-V4-Flash-Vision-Exp on SM120 (4x RTX PRO 6000 Blackwell 96 GB)

GLM-5.3-Flash on SM120 (4x RTX PRO 6000 Blackwell 96 GB)

Uh oh!

vLLM 0.28.1rc0 - DeepSeek-V4 Vision7 / GLM-5.3 SM89+SM120 (cu130/py3.12)

vLLM SM89 + SM120 Vision7 Wheels

Environment

Installation

DeepSeek-V4-Flash on SM89 (4x RTX 4090 48 GB)

DeepSeek-V4-Flash on SM120 (4x RTX PRO 6000 Blackwell 96 GB)

DeepSeek-V4-Flash-Vision-Exp on SM120 (4x RTX PRO 6000 Blackwell 96 GB)

GLM-5.3-Flash on SM120 (4x RTX PRO 6000 Blackwell 96 GB)

Uh oh!

vLLM 0.28.1rc0 - DeepSeek-V4 / GLM-5.3 SM89+SM120 (cu130/py3.12)

vLLM SM89 + SM120 Wheels

Environment

Installation

DeepSeek-V4-Flash on SM89 (4x RTX 4090 48 GB)

DeepSeek-V4-Flash on SM120 (4x RTX PRO 6000 Blackwell 96 GB)

GLM-5.3-Flash on SM120 (4x RTX PRO 6000 Blackwell 96 GB)

Uh oh!

vLLM 0.23.1rc1.dev904 - DeepSeek-V4 SM89 Upstream Sync (cu132/py3.12)

vLLM DeepSeek-V4 SM89 CUDA 13.2 Release

中文

用途

主要更新

构建环境

当前关键资产

安装

验证

English

Purpose

Highlights

Build Environment

Current Key Assets

Install

Validation

Uh oh!

vLLM 0.23.1rc1.dev904 - DeepSeek-V4-Flash-0731 SM89 Accuracy Fix (cu130/py3.12)

Purpose

Build Environment

Assets

Install

Notes

Validation

Uh oh!

vLLM 0.23.1rc1.dev904 - DeepSeek-V4 DSpark SM89 Issue #39 Fix (cu130/py3.12)

Purpose

Build Environment

Assets

Install

Notes

Validation

Uh oh!

vLLM 0.23.1rc1.dev1018 - DeepSeek-V4 DSpark SM89 FlashInfer 0.6.14 (cu130/py3.12)

Purpose

Build Environment

Assets

Install

Notes

Validation

Uh oh!

vLLM 0.23.1rc1.dev145 - DeepSeek-V4 DSpark SM80 testing build (cu130/py3.12)

Purpose

Build Environment

Asset