[Klaud Cold] [AMD] Bump DSV4 MI355X vLLM 8k/1k to the 2026-09-01 nightly / [Klaud Cold] [AMD] 将 DSV4 MI355X vLLM 8k/1k 更新至 2026-09-01 nightly

[jiacao-amd]

Summary

Update both fixed-sequence DeepSeek-V4-Pro FP4 MI355X vLLM configurations for the 8k/1k workload:

• dsv4-fp4-mi355x-vllm (STP)
• dsv4-fp4-mi355x-vllm-mtp (MTP)

Image update:

• From: vllm/vllm-openai-rocm:nightly-09663abde0f50944a8d5ea30120666024b503faa
• To: vllm/vllm-openai-rocm:nightly-7c5dc571cbd1064ecc8a9b1045637ff647aa22cb
• Digest: sha256:f0bdaf5217a09949842b45c1ea1f12260d3205ec81f143b320dfc2eb3ec95e55
• Docker Hub push time: 2026-09-01T05:31:02Z
• vLLM commit: 7c5dc571

The 8k/1k TP8 concurrency range remains unchanged at 4-512 for both STP and MTP.

vLLM DSV4 optimization review

The new nightly includes the following relevant changes merged after the 2026-08-17 nightly used by #2590:

• vLLM #52737: fuse DeepSeek-V4 mHC post/pre and RMSNorm with AITER.
• vLLM #52882: optimize DeepSeek-V4 C4A top-k with an automatic AITER/native hybrid selector on gfx950.
• vLLM #53838: fuse the DeepSeek-V4 C4 compressor GEMMs.
• vLLM #53540: fuse SWA q/kv RMSNorm and q FP8 group quant on the ROCm/AITER path.
• vLLM #53382: tune cooperative top-k for medium batch sizes.

The newly merged DSV4 kernels are selected automatically through the existing VLLM_ROCM_USE_AITER=1, VLLM_ROCM_USE_AITER_MOE=1, and --moe-backend aiter settings. For parity with the latest ROCm recipe, both STP and MTP now also set:

• VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1. The current deepseek-ai/DeepSeek-V4-Pro checkpoint exposes FP4 routed experts with an FP8 block-quantized shared-expert path, so the latest vLLM compatibility check self-disables this fusion when the path is not eligible.
• VLLM_ROCM_QUICK_REDUCE_QUANTIZATION=INT4 to enable INT4 quantized quick all-reduce; the full sweep evals cover the accuracy impact.

Validation

• Verified the immutable tag and digest through Docker Hub metadata.
• Verified the listed upstream commits are ancestors of the pinned vLLM commit.
• Generated the targeted matrix successfully with 8 STP and 8 MTP points:

/home/jiacao/.local/bin/uv run --with 'pydantic>=2' --with pyyaml \
  python3 utils/matrix_logic/generate_sweep_configs.py full-sweep \
  --config-files configs/amd-master.yaml \
  --model-prefix dsv4 \
  --framework vllm \
  --precision fp4 \
  --runner-type mi355x \
  --min-conc 4 --max-conc 512 \
  --seq-lens 8k1k --no-evals

• git diff --check passes.
• GPU sweep pending for the latest commit.

中文说明

更新 MI355X 上 DeepSeek-V4-Pro FP4 的两条固定序列 8k/1k vLLM 配置：

• dsv4-fp4-mi355x-vllm（STP）
• dsv4-fp4-mi355x-vllm-mtp（MTP）

镜像变更：

• 原镜像：vllm/vllm-openai-rocm:nightly-09663abde0f50944a8d5ea30120666024b503faa
• 新镜像：vllm/vllm-openai-rocm:nightly-7c5dc571cbd1064ecc8a9b1045637ff647aa22cb
• Digest：sha256:f0bdaf5217a09949842b45c1ea1f12260d3205ec81f143b320dfc2eb3ec95e55
• Docker Hub 推送时间：2026-09-01T05:31:02Z
• vLLM 提交：7c5dc571

STP 和 MTP 的 8k/1k TP8 并发范围均保持 4-512 不变。

vLLM DSV4 优化核对

与 #2590 使用的 2026-08-17 nightly 相比，新镜像包含以下相关优化：

• vLLM #52737：通过 AITER 融合 DeepSeek-V4 mHC post/pre 与 RMSNorm。
• vLLM #52882：在 gfx950 上为 DeepSeek-V4 C4A top-k 自动选择 AITER/native 混合路径。
• vLLM #53838：融合 DeepSeek-V4 C4 compressor GEMM。
• vLLM #53540：在 ROCm/AITER 路径融合 SWA q/kv RMSNorm 与 q FP8 group quant。
• vLLM #53382：优化中等 batch size 下的 cooperative top-k。

新合入的 DSV4 kernel 会通过现有的 VLLM_ROCM_USE_AITER=1、VLLM_ROCM_USE_AITER_MOE=1 和 --moe-backend aiter 自动选用。为了与最新 ROCm 配方保持一致，STP 和 MTP 现在还会显式设置：

• VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1。当前 deepseek-ai/DeepSeek-V4-Pro 检查点采用 FP4 routed experts 与 FP8 block-quantized shared-expert 路径；当该路径不满足条件时，最新 vLLM 的兼容性检查会自动关闭此融合。
• VLLM_ROCM_QUICK_REDUCE_QUANTIZATION=INT4，启用 INT4 量化 quick all-reduce；完整扫描中的准确率评估会覆盖其精度影响。

中文验证

• 已通过 Docker Hub 元数据核对不可变 tag 与 digest。
• 已确认上述上游提交均包含在固定的 vLLM 提交中。
• 已成功生成目标矩阵，共 8 个 STP 点和 8 个 MTP 点；命令见英文部分。
• git diff --check 通过。
• 最新提交的 GPU 完整扫描等待执行。

---

Note

Medium Risk
Benchmark numbers and accuracy may shift from a new vLLM nightly plus INT4 quick all-reduce; scope is limited to MI355X DSV4 FP4 vLLM STP/MTP configs and launch env, not core serving code.

Overview
Bumps the pinned vllm/vllm-openai-rocm image for dsv4-fp4-mi355x-vllm and dsv4-fp4-mi355x-vllm-mtp from nightly-09663abde0f50944a8d5ea30120666024b503faa to nightly-7c5dc571cbd1064ecc8a9b1045637ff647aa22cb (2026-09-01 nightly). TP8 concurrency sweep (4–512) is unchanged; config comments now describe AITER + explicit recipe env vars instead of the older flag list.

Aligns the STP and MTP fixed-seq-len launch scripts with the current ROCm recipe by exporting VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1 and VLLM_ROCM_QUICK_REDUCE_QUANTIZATION=INT4 alongside the existing AITER MoE settings. Shared-expert fusion is expected to no-op for this checkpoint when ineligible.

Documents the refresh in perf-changelog.yaml for both config keys, including the new digest and the upstream DSV4 kernel optimizations picked up in the image.

^{Reviewed by Cursor Bugbot for commit 77dd5ab. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[claude]

Claude finished @jiacao-amd's task in 1m 56s —— View job

---

PR Review

• Gather context and read PR diff
• Verify perf-changelog entry is appended at end of file
• Verify image tag/digest and config consistency
• Validate config recipe claims (AITER flags, concurrency range)
• Post review

LGTM - no blocking issues found.

Verification details:

• configs/amd-master.yaml was modified and perf-changelog.yaml was updated in the same PR, with the new entry correctly appended at the end of the file (lines 6734-6743) referencing both config keys and this PR link. ✓
• The diff is behaviorally minimal: only the two image: lines change (nightly-09663abde... → nightly-7c5dc571cb...); everything else is comment updates. Both STP and MTP keep the TP8 conc-start: 4, conc-end: 512 search space unchanged, matching the PR description. ✓
• The image uses the pinned digest-suffixed tag convention on the public vllm/vllm-openai-rocm Docker Hub repo (required by the enroot squashfs cache noted in the config comment — the floating :nightly tag would silently reuse a stale build). I could not independently query Docker Hub from this environment (no network permission), but the tag format and the digest documented in the PR body follow the established pattern. ✓
• The changelog's claim that no new serving flags are needed checks out: benchmarks/single_node/fixed_seq_len/dsv4_fp4_mi355x_vllm.sh:47-48 and dsv4_fp4_mi355x_vllm_mtp.sh:42-43 already export VLLM_ROCM_USE_AITER=1 / VLLM_ROCM_USE_AITER_MOE=1 and pass --moe-backend aiter, and neither script was modified. Expert parallelism remains correctly gated on EP_SIZE. ✓
• The rewritten comment block in configs/amd-master.yaml removes a stale description ("TP8 sweeps conc 4-64; DEP8 has a single conc=64 probe") that no longer matched the actual 4-512 search space — a correct cleanup.

[claude]

Looks good, straightforward image bump. Reviewed the amd-master.yaml diff (only the two image: tags and their explanatory comments change; concurrency ranges, TP, and scenario fields are untouched) and confirmed the perf-changelog.yaml entry is appended at the tail per the append-only convention, matching the two config keys touched.

Extended reasoning...

Overview

The diff touches only configs/amd-master.yaml and perf-changelog.yaml. In amd-master.yaml, the two dsv4-fp4-mi355x-vllm and dsv4-fp4-mi355x-vllm-mtp entries have their image: field bumped from one ROCm nightly tag to another (both digest-suffixed, per the repo's stated digest-pinning requirement), plus rewritten explanatory comments describing why no new serving flags are needed. No other YAML fields (model, precision, framework, search-space/concurrency ranges, spec-decoding) changed. perf-changelog.yaml gets one new entry appended at the very tail, listing the two config keys, scenario-type, old/new image tags, digest, and a note that existing AITER env vars/flags are unchanged — consistent with AGENTS.md's append-only, byte-sensitive changelog rule.

Security risks

None. This is a version/tag bump in a YAML config plus a changelog entry; no code execution paths, auth, or secrets are touched.

Level of scrutiny

Low. This is exactly the class of change AGENTS.md and the guidelines call out as approvable: a mechanical image-tag bump with matching documentation update, no serving-flag or scenario changes in the YAML body itself. The model.container must equal image rule from AGENTS.md applies only to multi-node srt-slurm recipes, which is not the case here (these entries have no model.container field), so that invariant is inapplicable rather than violated.

Other factors

The bug hunter reported zero findings. I independently verified via git diff that the only body (non-comment) changes are the two image: lines, and that the perf-changelog entry was appended after the prior tail entry (PR #2634) without altering any preceding bytes. No CODEOWNERS check was needed since this is a generic recipe config, and the PR description's claims (digest, upstream vLLM commits) are not independently verifiable from the diff alone, but they don't affect the correctness of the config-file mechanics being reviewed here.

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33478188035
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=33478188035

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33480422577
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=33480422577

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33537691400
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=33537691400

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33681446902
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=33681446902