[AMD][MI35X] Bump Qwen3.5 MXFP4 MI355X SGLang single-node image to v0.5.17 - #2640
[AMD][MI35X] Bump Qwen3.5 MXFP4 MI355X SGLang single-node image to v0.5.17#2640yichiche wants to merge 2 commits into
Conversation
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
2 similar comments
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
dfab028 to
ff174eb
Compare
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as outdated.
This comment was marked as outdated.
090eb36 to
0ffaead
Compare
0ffaead to
7c92949
Compare
This comment was marked as off-topic.
This comment was marked as off-topic.
7c92949 to
2a47c0a
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32212675501 |
|
Cookbook: sgl-project/sglang#35445 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32241233599 |
…m3 KV cache Match the fp8_e4m3 KV cache that #2640 applies to the fixed-seq-len arms. The other MTP tunings from that PR are already present in the agentic script (mamba ssm dtype, 32768-token prefill/chunked prefill, tokenizer workers) or are deliberately different for long-running trace replay (scheduler-recv-interval 30, stream-interval 50).
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32275897225 |
…-20260818 (#2677) * [AMD][MI35X] Bump Qwen3.5 MXFP4 MI355X SGLang AgentX image to v0.5.17-20260818 Bump the qwen3.5-fp4-mi355x-sglang-agentic-mtp image from lmsysorg/sglang-rocm:v0.5.17-rocm720-mi35x-20260811 to lmsysorg/sglang-rocm:v0.5.17-rocm720-mi35x-20260818 and append the matching agentic-coding perf-changelog trigger. * [AMD][MI35X] Point Qwen3.5 AgentX changelog entry to PR #2677 * [AMD][MI35X] Serve the Qwen3.5 MXFP4 MI355X AgentX arm with an fp8_e4m3 KV cache Match the fp8_e4m3 KV cache that #2640 applies to the fixed-seq-len arms. The other MTP tunings from that PR are already present in the agentic script (mamba ssm dtype, 32768-token prefill/chunked prefill, tokenizer workers) or are deliberately different for long-running trace replay (scheduler-recv-interval 30, stream-interval 50). * [AMD][MI35X] Restore the trailing newline in perf-changelog.yaml The merge of main into this branch left perf-changelog.yaml without a final newline, which check-changelog rejects. --------- Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
….5.17 Bump qwen3.5-fp4-mi355x-sglang and qwen3.5-fp4-mi355x-sglang-mtp from lmsysorg/sglang-rocm:v0.5.16-rocm720-mi35x-20260730 to lmsysorg/sglang-rocm:v0.5.17-rocm720-mi35x-20260818, serve both arms with an fp8_e4m3 KV cache, and append the matching perf-changelog trigger.
79083bc to
bfddbcf
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32367046513 |
…X MTP arm Add --disable-shared-experts-fusion to qwen3.5_fp4_mi355x_mtp.sh. The non-MTP arm is unchanged.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32367711648 |
1 participant
Motivation
qwen3.5-fp4-mi355x-sglangandqwen3.5-fp4-mi355x-sglang-mtpare still pinned tolmsysorg/sglang-rocm:v0.5.16-rocm720-mi35x-20260730, which is roughly three weeks behind the current SGLang ROCm nightly. This bumps both single-node MI355X Qwen3.5 MXFP4 arms to the latestv0.5.17mi35x image so the benchmark matrix reflects current upstream SGLang + aiter performance on MI355X.Modifications
Bump the
image:value for both config keys inconfigs/amd-master.yamlfromlmsysorg/sglang-rocm:v0.5.16-rocm720-mi35x-20260730tolmsysorg/sglang-rocm:v0.5.17-rocm720-mi35x-20260817. No other config fields change — model, runner, TP/EP search space, and concurrency ranges are untouched.Append the corresponding trigger entry to
perf-changelog.yaml(append-only, at the end of the file) covering both config keys.Benchmark scripts under
benchmarks/single_node/fixed_seq_len/are unchanged; this is a plain image bump.Accuracy Tests
No accuracy-affecting logic changes in this repo — the change is limited to the container tag and the changelog trigger. Model correctness is covered by the upstream SGLang release validation for the
v0.5.17ROCm image.Benchmarking
Repo validation was run locally:
python -m pytest utils/matrix_logic/ -q→ 231 passed.python utils/matrix_logic/generate_sweep_configs.py full-sweep --config-files configs/amd-master.yaml --model-prefix qwen3.5 --precision fp4 --runner-type mi355xemits the new tag on both arms: 10 generated configs withspec-decoding: noneand 9 withspec-decoding: mtp, all referencinglmsysorg/sglang-rocm:v0.5.17-rocm720-mi35x-20260817.End-to-end MI355X throughput/latency numbers will come from the sweep triggered on this PR (
full-sweep-fail-fast).