feat(minimaxm3-fp4-b300-dynamo-vllm-mtp-legacy-dep4): bump vLLM nightly to 5e35a6f4, enable cutlass MSA decode / 更新 minimaxm3-fp4-b300-dynamo-vllm-mtp-legacy-dep4 vLLM 镜像至 5e35a6f4，启用 cutlass MSA 解码后端 - #2483
Conversation
…ly to 5e35a6f4, enable cutlass MSA decode 中文：将 minimaxm3-fp4-b300-dynamo-vllm-mtp-legacy-dep4 vLLM 镜像升级至 nightly-5e35a6f4f9bbc217c599692157ca985c894373f7，在解码侧 attention-config 中启用 minimax_m3_msa_decode_backend=cutlass Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
…p-legacy-dep4 nightly bump Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
LGTM — straightforward vLLM nightly image bump with matching attention-config changes for the minimax-m3 b300-fp4 dep4 recipe.
What was reviewed: confirmed the container tag bump is consistent across the recipe YAML and configs/nvidia-master.yaml, checked the new minimax_m3_msa_decode_backend=cutlass and kv-cache-dtype: fp8 additions are scoped to decode/prefill attention-config as described, and the perf-changelog entry matches the PR changes.
Extended reasoning...
Overview
This PR bumps the vLLM nightly image for minimaxm3-fp4-b300-dynamo-vllm-mtp-legacy-dep4 from nightly-8e981630 to nightly-5e35a6f4, adds kv-cache-dtype: fp8 to both prefill and decode vllm_config blocks, and enables minimax_m3_msa_decode_backend=cutlass in the decode attention-config string. Three files are touched: the benchmark recipe YAML, configs/nvidia-master.yaml (image tag), and perf-changelog.yaml (changelog entry). This follows the repo's standard pattern for image-bump PRs.
Security risks
None. This is a benchmark configuration change (container tag, kv-cache dtype, attention backend flag) with no code, auth, or data-handling logic involved.
Level of scrutiny
Low. This is a mechanical, well-scoped config change consistent with many prior nightly-bump PRs in this repo. The image tag is updated identically in both the recipe YAML and the master config, and the new attention-config/kv-cache-dtype fields are additive and match the PR description exactly.
Other factors
The bug-hunting system found no issues, and the one candidate concern raised (a perceived missing perf-changelog entry) was checked and is present in the diff. The PR carries the full-sweep-fail-fast label so the actual runtime behavior of the new image/backend combination will be validated by the benchmark sweep before merge.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30860526083 |
|
/reuse-sweep-run |
kedarpotdar-nv
left a comment
kedarpotdar-nv
left a comment
There was a problem hiding this comment.
As a PR reviewer and CODEOWNER, I have reviewed this and have:
- Verified that as of the moment of typing this, this is the latest version of PR_REVIEW_CHECKLIST.md
- Verified that the general code quality meets the InferenceX standard and does not make the code quality any worse.
- Verified that this PR has passed PR validation. Please link to GitHub Action workflow that shows this.
- Verified that this PR passes evals. Please link to GitHub Action workflow that shows this.
- Verified that speculative decoding PRs uses chat templates to align the AL distribution to real world
- For agentic workloads: verified that speculative-decoding configs (EAGLE / MTP / draft models) run with simulated synthetic acceptance, with the acceptance-length value taken from the committed golden AL curve in golden_al_distribution/ for that model, thinking mode, and draft length. A submission may choose any supported draft length, but it may not substitute a different acceptance target.
- Verified against the current MODELS.md that this PR does not submit a deprecated model, scenario, or model-scenario combination.
- Verified that the model architecture isn't changed with benchmark hacks like using --hf-overrides to skipping indexer for every x layers on models that don't natively support this. As a general rule, we won't accept optimizations that reduces the number of model architecture FLOPs. Anything that makes that same computation run faster is fair game; FLOPs at lower precisions is fine, given that the config passes private evals. As an general north star princple, we should only use optimizations which is used in production by customers that care about accuracy
- If an company claims that they support vLLM/SGLang as first class LLM inference engines on their hardware, I have verified that the respective vLLM submission made using upstream https://hub.docker.com/u/vllm docker repo, upstream SGLang https://hub.docker.com/u/lmsysorg docker repo. The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet as supported by vLLM/SGLang community maintainers
- If an company claims that they support vLLM/SGLang as first class upstream in-tree LLM inference engines on their hardware, I have have verified that the respective vLLM/SGLang submission has been made before additional frameworks (TRT-LLM, ATOM, etc.). The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet.
- Verified that every single-node vLLM/SGLang recipe in this PR is documented in the official vLLM recipes and/or the SGLang cookbook:
- I linked the corresponding upstream PR in the vLLM recipe repo or SGLang repo and verified that it is MERGED before this InferenceX PR merges. An opened, draft, or closed-without-merge upstream PR does not satisfy this requirement. If the matching recipe was already published, I linked the published recipe/cookbook page in the additional detail section below.
- Verified that this PR does not patch the inference engine or serving stack — the pinned image must run as shipped. This covers .patch files / git apply / patch, inline patches embedded in benchmark scripts (e.g. a python3/sed heredoc that rewrites installed engine sources before serving), in-place edits of site-packages, monkey-patching, overwriting container files, and installing forked/rebuilt engine wheels on top of the pinned image. The only exception is a patch covered by a filled-out waiver at docs/waiver/
<PR_NUMBER>.md— named after the PR that introduces the patch and filed in that same PR, stating what is patched, why the unmodified upstream image cannot run this benchmark, the upstream PR/issue link, and the removal plan — which I have linked below in the additional detail section. - If any of the above criteria cannot reasonably be satisfied, I have provided additional reasoning below.
Additional detail section:
- Validation and evals: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/30860526083
- Speculative decoding uses a chat template in the updated multi-node recipe.
- Upstream recipe check is N/A because this PR changes only a disaggregated multi-node recipe.
- Uses the upstream
vllm/vllm-openaiimage; no serving-stack patches or model-architecture overrides are introduced. - Authorized sweep reuse: #2483 (comment)
Signed: kedarpotdar-nv
❌❌❌ REJECTED ❌❌❌@kedarpotdar-nv — blocking issue: this PR changes a deprecated model-scenario combination. ✅ Check 0 (CODEOWNER): PASS — |
中文：将 main 合并到 MiniMax M3 B300 更新分支并解决性能变更日志冲突。
中文：解决仅追加性能变更日志的合并冲突。
4 participants
Summary
minimaxm3-fp4-b300-dynamo-vllm-mtp-legacy-dep4(conc 256, 512) vLLM image tonightly-5e35a6f4f9bbc217c599692157ca985c894373f72p1d-dep2-dep4-eagle3-8k1k.yaml: new container,minimax_m3_msa_decode_backend=cutlassin decode attention-config,kv-cache-dtype: fp8in prefill and decodeTest plan
full-sweep-fail-fastlabel runs green at conc 256 and 512中文说明
minimaxm3-fp4-b300-dynamo-vllm-mtp-legacy-dep4（并发 256、512）vLLM 镜像升级至nightly-5e35a6f4f9bbc217c599692157ca985c894373f72p1d-dep2-dep4-eagle3-8k1k.yaml：同步 container 字段，在解码侧 attention-config 中启用minimax_m3_msa_decode_backend=cutlass，在预填充与解码配置中添加kv-cache-dtype: fp8🤖 Generated with Claude Code