[Handoff to @Oseltamivir Claude /loop] [Klaud Cold] Add dsr1-fp8-mi355x-sglang-mtp single-node MTP recipe#1521
Conversation
New MTP/EAGLE speculative-decoding sibling for the existing dsr1-fp8-mi355x-sglang recipe. Recipe key: dsr1-fp8-mi355x-sglang-mtp Model: deepseek-ai/DeepSeek-R1-0528 (same as sibling) Image: lmsysorg/sglang:v0.5.12-rocm700-mi35x (same as sibling) Search: tp=8 ep=1 conc 4..64 spec-decoding=mtp on 1k1k + 8k1k Launch script benchmarks/single_node/dsr1_fp8_mi355x_mtp.sh clones the off variant (dsr1_fp8_mi355x.sh) and adds: --ep-size $EP_SIZE --speculative-algorithm EAGLE --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 plus SGLANG_AITER_MLA_PERSIST=1 / SGLANG_ENABLE_SPEC_V2=1 env vars and --use-chat-template on the bench client (mirrors dsr1_fp8_mi325x_mtp.sh, the production sglang MTP template). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
functionstackx
requested review from
1am9trash,
billishyahao,
chunfangamd,
seungrokj and
yctseng0211
as code owners
|
Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers. If additional help is needed, PR authors can reach out to core maintainers over Slack. |
|
Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers. If additional help is needed, PR authors can reach out to core maintainers over Slack. |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
LGTM — additive single-node MTP recipe that mirrors the existing dsr1_fp8_mi325x_mtp.sh template.
Extended reasoning...
Overview
The PR adds a new dsr1-fp8-mi355x-sglang-mtp recipe entry consisting of three small additions: a YAML config block in .github/configs/amd-master.yaml, a new launch script benchmarks/single_node/dsr1_fp8_mi355x_mtp.sh, and a corresponding entry in perf-changelog.yaml. The change is purely additive — no existing recipes or scripts are modified.
Security risks
None. This PR contains only benchmark configuration data and a shell launch script that runs within an isolated benchmark container. No auth, crypto, permissions, networking changes, or anything else security-sensitive is touched.
Level of scrutiny
Low. This is a routine recipe addition that follows a well-established pattern: it clones dsr1_fp8_mi355x.sh and adds the same EAGLE speculative-decoding flags, env vars (SGLANG_AITER_MLA_PERSIST=1, SGLANG_ENABLE_SPEC_V2=1), --ep-size, and --use-chat-template that the production dsr1_fp8_mi325x_mtp.sh recipe already uses. The YAML block matches the structure of the sister entries (qwen3.5-bf16-mi355x-sglang-mtp, dsr1-fp8-mi325x-sglang-mtp, etc.) in the same file.
Other factors
The bug-hunting system found no issues. The PR includes a clear test plan (YAML loads, bash -n passes), and the full-sweep-enabled label will exercise the sweep on CI. Worst case for a bad recipe is a failing sweep that gets re-run or reverted — it cannot impact other recipes or production code paths.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26134317173 |
2 similar comments
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26134317173 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26134317173 |
functionstackx
changed the title
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26134317173 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26197428678 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26197707118 |
|
/reuse-sweep-run |
2 participants
Summary
Add the single-node MTP/EAGLE speculative-decoding sibling for the existing
dsr1-fp8-mi355x-sglangrecipe.dsr1-fp8-mi355x-sglang-mtpdeepseek-ai/DeepSeek-R1-0528(same as the off sibling)lmsysorg/sglang:v0.5.12-rocm700-mi35x(same as the off sibling)tp=8 ep=1 conc 4..64 spec-decoding=mtpon 1k1k + 8k1kLaunch script
benchmarks/single_node/dsr1_fp8_mi355x_mtp.shclones the off variant (dsr1_fp8_mi355x.sh) and adds, modeled on the productiondsr1_fp8_mi325x_mtp.shtemplate:--ep-size $EP_SIZE--speculative-algorithm EAGLE --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4SGLANG_AITER_MLA_PERSIST=1,SGLANG_ENABLE_SPEC_V2=1--use-chat-templateon the bench clientTest plan
bash -nsyntax passes on the new launch script.