Add DSv4 FP8 H200 vLLM MTP benchmark

[functionstackx]

Summary

Ports the H200 STP recipe to MTP.

• New dsv4-fp8-h200-vllm-mtp config + benchmarks/single_node/dsv4_fp8_h200_mtp.sh script.
• MTP counterpart of dsv4-fp8-h200-vllm: identical launch flags (EP + DP=$TP, --gpu-memory-utilization 0.95, --max-num-seqs 512, --no-enable-flashinfer-autotune, FULL_DECODE_ONLY compile) with one addition — --speculative-config '{"method":"mtp","num_speculative_tokens":1}'.
• num_speculative_tokens=1 because the recipe states H200 supports spec token=1 only for now (B200 / Blackwell SKUs are where token=2 lands on the Pareto front per @wzhao18's vLLM Blackwell MTP work; the H200 kernel coverage is still token=1).
• Image: vllm/vllm-openai:v0.20.1 (pinned by SHA256 digest 9eff9734...23aa4). The non-MTP dsv4-fp8-h200-vllm entry is unchanged and still on the deepseekv4-cu129 tag.
• max-model-len comes from the runner's $MAX_MODEL_LEN env var rather than being hardcoded to 800k — keeps this script consistent with the rest of the H200 fleet.
• Sets VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS=0 to skip the cudagraph-memory estimator during worker memory profiling — it overestimates and pushes us over the GPU memory budget on H200 + MTP, even though the actual cudagraph capture works fine.
• run_benchmark_serving invocation passes --dsv4 so prompts get chat-formatted encoding, per the AGENTS.md MTP rule (raw random tokens silently regress EAGLE-style acceptance).
• Search space mirrors the non-MTP H200 entry: TP=8, EP=8, DP-attn=true, CONC 4-64, both 1k1k and 8k1k, with spec-decoding: mtp on each entry.
• Adds a perf-changelog.yaml entry to trigger the new config.

Test plan

• Trigger the dsv4-fp8-h200-vllm-mtp benchmark workflow on an H200 runner and confirm the engine starts and the sweep completes for at least one cell from each of the two seq-len-configs.
• Confirm vllm/vllm-openai:v0.20.1@sha256:9eff9734...23aa4 pulls cleanly.
• server.log shows --speculative-config '{"method":"mtp","num_speculative_tokens":1}' and the rest of the H200 recipe flags.
• Acceptance rate is in a sane range — --dsv4 is wired into run_benchmark_serving so the prompts go through the chat template.

🤖 Generated with Claude Code

[github-actions]

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

• https://cookbook.sglang.io/
• https://docs.vllm.ai/projects/recipes/en/latest/

If it is not, please create a PR first before we can merge your PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

• https://github.com/vllm-project/recipes
• https://github.com/sgl-project/sgl-cookbook

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow

As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers.

If additional help is needed, PR authors can reach out to core maintainers over Slack.

[claude]

🔴 The MTP benchmark script is added at benchmarks/single_node/dsv4_fp8_h200_vllm_mtp.sh, but all three H200 launch scripts (runners/launch_h200-cw.sh:47, runners/launch_h200-nb.sh:22, runners/launch_h200-dgxc-slurm.sh:295) build the script path as benchmarks/single_node/${MODEL_CODE}_${PRECISION}_h200${FRAMEWORK_SUFFIX}${SPEC_SUFFIX}.sh where FRAMEWORK_SUFFIX is empty for vllm — so they will look for dsv4_fp8_h200_mtp.sh and fail with 'No such file or directory' on every cell of the sweep. Unlike launch_b300-nv.sh, the H200 launchers have no framework-tagged-name fallback. Fix by either renaming the script to dsv4_fp8_h200_mtp.sh (matches the existing convention — see qwen3.5_fp8_h200_mtp.sh) or porting the B300 fallback logic to the H200 launchers.

Extended reasoning...

What the bug is

The PR adds a new vLLM MTP benchmark script at benchmarks/single_node/dsv4_fp8_h200_vllm_mtp.sh and a corresponding dsv4-fp8-h200-vllm-mtp config in .github/configs/nvidia-master.yaml. However, the filename does not match what the H200 launch scripts will look for at runtime, so the workflow will hard-fail before vLLM ever starts.

How the launcher resolves the script path

All three H200 launch scripts build the benchmark script path the same way:

# runners/launch_h200-cw.sh:7-8, 47
MODEL_CODE="${EXP_NAME%%_*}"
FRAMEWORK_SUFFIX=$([[ "$FRAMEWORK" == "trt" ]] && printf '_trt' || printf '')
SPEC_SUFFIX=$([[ "$SPEC_DECODING" == "mtp" ]] && printf '_mtp' || printf '')
...
bash benchmarks/single_node/${MODEL_CODE}_${PRECISION}_h200${FRAMEWORK_SUFFIX}${SPEC_SUFFIX}.sh

runners/launch_h200-nb.sh:7-8,22 is identical, and runners/launch_h200-dgxc-slurm.sh:295 inlines the same construction.

FRAMEWORK_SUFFIX is _trt only when the framework is trt; for vllm (and sglang) it is empty. SPEC_SUFFIX is _mtp when SPEC_DECODING=mtp.

Step-by-step proof for the new config

For the new dsv4-fp8-h200-vllm-mtp entry:

variable	value
model-prefix	dsv4
MODEL_CODE	dsv4 (from EXP_NAME="${model_code}_${seq_len_str}")
PRECISION	fp8
FRAMEWORK	vllm → FRAMEWORK_SUFFIX=""
SPEC_DECODING	mtp → SPEC_SUFFIX="_mtp"

So the resolved path is:

benchmarks/single_node/dsv4_fp8_h200_mtp.sh

But the PR added the file at:

benchmarks/single_node/dsv4_fp8_h200_vllm_mtp.sh

bash will exit with No such file or directory, the runner will mark the cell as failed, and every cell of the new sweep (TP=8/EP=8, conc 4–64, both 1k1k and 8k1k) will fail before the engine starts.

Why the existing code does not save it

Unlike runners/launch_b300-nv.sh:267-272, which prefers a framework-tagged name and falls back to the legacy un-tagged name (this is exactly why dsv4_fp4_b300_vllm_mtp.sh and dsv4_fp4_b300_sglang_mtp.sh work on B300), the H200 launchers have no fallback — they construct one path and run it.

The existing H200 file naming convention confirms the expected name: every other vLLM/SGLang H200 MTP/non-MTP script in the tree omits the framework name (qwen3.5_fp8_h200_mtp.sh, dsr1_fp8_h200.sh, glm5_fp8_h200.sh, dsv4_fp8_h200.sh from this same series), and the only framework-tagged H200 scripts use _trt (dsr1_fp8_h200_trt_mtp.sh). The non-MTP counterpart in this PR's series — dsv4_fp8_h200.sh — already follows the no-suffix convention and works, which is itself evidence of the bug.

Impact and fix

This is a hard, deterministic PR-blocker: every cell of the new benchmark sweep fails to launch. Two fixes:

1. Simplest: rename benchmarks/single_node/dsv4_fp8_h200_vllm_mtp.sh → benchmarks/single_node/dsv4_fp8_h200_mtp.sh to match the existing H200 convention.
2. Or: port the B300-style framework-tagged-then-legacy-fallback logic to all three H200 launch scripts so framework-tagged filenames also work.

[functionstackx]

waiting for 0.20.1 vllm-project/vllm#41189 and vllm-project/vllm#41444

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=25344464277
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=25344464277