(ci)(recipe): Add DeepSeek-R1 FP4 TP4 validation and DS recipe for SGLang-ATOM

[zhuyuhua-v]

Motivation

• add DeepSeek-R1-FP4 TP4 coverage to SGLang-ATOM accuracy flows, including nightly/manual validation and dashboard metadata, with a 0.91 GSM8K threshold
• align the DeepSeek-R1-FP8 TP4 GSM8K threshold to 0.91 across the ATOM SGLang PR and nightly accuracy workflows to avoid data floating issues.
• add recipes/sglang_atom/DeepSeek-R1.md in the same style as the vLLM-ATOM recipe, covering server launch, benchmarking, accuracy validation, and profiling usage
• Updates aiter wheel download, align with PR [atom-vllm CI] align the aiter download logic with atom CI #706

ATOM SGLang CI / Nightly / Benchmark Scope

Scope	Workflow	Trigger	Case 数	用途
CI	.github/workflows/atom-sglang-test.yaml	PR to main，非 draft，非 closed	2	PR SGLang accuracy smoke
Nightly Accuracy	.github/workflows/atom-sglang-accuracy-validation.yaml	每天 18:00 UTC / 北京 02:00，或手动触发	4	全量 SGLang GSM8K accuracy validation
Nightly Benchmark	.github/workflows/atom-sglang-benchmark.yaml	每天 17:00 UTC / 北京 01:00，或手动触发	nightly: 5 × 10 = 50	SGLang serving performance benchmark

Shared Accuracy Parameters

Item	Value
SGLang ref	v0.5.10
Task	gsm8k
Metric checked	results.gsm8k["exact_match,flexible-extract"]
Few-shot	3
LM Eval concurrency	65
Server args	--trust-remote-code --kv-cache-dtype fp8_e4m3 --mem-fraction-static 0.8 --page-size 1 --disable-radix-cache
Common env	SGLANG_AITER_FP8_PREFILL_ATTN=0, SGLANG_USE_AITER=1, ATOM_ENABLE_DS_QKNORM_QUANT_FUSION=1

CI Cases

Model	Weight	Runner	TP	Extra Args	Env Vars	Threshold
DeepSeek-R1-FP8 TP4	deepseek-ai/DeepSeek-R1-0528	linux-atom-mi35x-4	4	--tensor-parallel-size 4	AITER_QUICK_REDUCE_QUANTIZATION=INT4; common env	0.91
DeepSeek-R1-FP4 TP4	amd/DeepSeek-R1-0528-MXFP4-MTP-MoEFP4	linux-atom-mi35x-4	4	--tensor-parallel-size 4	AITER_QUICK_REDUCE_QUANTIZATION=INT4; common env	0.91

Nightly Accuracy Cases

Model	Weight	Runner	TP	Extra Args	Threshold
DeepSeek-R1-FP8 TP4	deepseek-ai/DeepSeek-R1-0528	linux-atom-mi35x-4	4	--tensor-parallel-size 4	0.91
DeepSeek-R1-FP8 TP8	deepseek-ai/DeepSeek-R1-0528	linux-atom-mi35x-8	8	--tensor-parallel-size 8	0.93
DeepSeek-R1-FP4 TP4	amd/DeepSeek-R1-0528-MXFP4-MTP-MoEFP4	linux-atom-mi35x-4	4	--tensor-parallel-size 4	0.91
DeepSeek-R1-FP4 TP8	amd/DeepSeek-R1-0528-MXFP4-MTP-MoEFP4	linux-atom-mi35x-8	8	--tensor-parallel-size 8	0.93

Benchmark Schedule

当前 benchmark workflow 支持两种模式：

Mode	Model Selection	Param Selection	Dashboard
Scheduled nightly	自动选择全部 5 个 SGLang benchmark models	默认 10 组参数	默认 publish
Manual dispatch	通过 checkbox 选择模型	param_lists 输入，默认 10 组参数	publish_to_dashboard 控制，默认 true

Schedule:

• Cron: 0 17 * * *
• Beijing time: 每晚 01:00

Benchmark Parameters

Default param sets:

ISL	OSL	Concurrency	Random Range Ratio
1024	1024	4, 8, 16, 32, 64	0.8
8192	1024	4, 8, 16, 32, 64	0.8

Benchmark command:

• backend: sglang
• dataset: random
• num-prompts = concurrency * 10
• num-warmups = 2 * concurrency
• request-rate=inf
• metrics: ttft,tpot,itl,e2el

Benchmark Models

Model	Weight	Serve Args	Runner
DeepSeek-R1-0528 FP8 TP8	deepseek-ai/DeepSeek-R1-0528	--trust-remote-code --tensor-parallel-size 8	atom-mi355-8gpu-oot-benchmark
DeepSeek-R1-0528 FP8 TP4	deepseek-ai/DeepSeek-R1-0528	--trust-remote-code --tensor-parallel-size 4	atom-mi355-8gpu-oot-benchmark
DeepSeek-R1-0528-MXFP4 FP4 TP8	amd/DeepSeek-R1-0528-MXFP4-MTP-MoEFP4	--trust-remote-code --tensor-parallel-size 8	atom-mi355-8gpu-oot-benchmark
DeepSeek-R1-0528-MXFP4 FP4 TP4	amd/DeepSeek-R1-0528-MXFP4-MTP-MoEFP4	--trust-remote-code --tensor-parallel-size 4	atom-mi355-8gpu-oot-benchmark
DeepSeek-R1-0528-MXFP4 FP4 TP8 EP8	amd/DeepSeek-R1-0528-MXFP4-MTP-MoEFP4	--trust-remote-code --tensor-parallel-size 8 --expert-parallel-size 8	atom-mi355-8gpu-oot-benchmark

[valarLip]

still wip?

[Copilot]

Pull request overview

Adds DeepSeek-R1 FP4 (MXFP4 weights) TP4 accuracy coverage to the ATOM SGLang CI/validation flows and documents how to run/benchmark/validate DeepSeek-R1 using the SGLang-ATOM backend.

Changes:

• Add DeepSeek-R1 FP4 TP4 (MXFP4 checkpoint) to PR CI accuracy matrix and to nightly/manual accuracy validation matrix.
• Align DeepSeek-R1 FP8 TP4 GSM8K accuracy threshold from 0.92 to 0.91 across workflows and dashboard model metadata.
• Add an SGLang-ATOM DeepSeek-R1 recipe covering server launch, benchmarking, profiling, and GSM8K validation.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File	Description
recipes/sglang_atom/DeepSeek-R1.md	New SGLang-ATOM DeepSeek-R1 recipe (launch, benchmark, profiling, lm-eval).
.github/workflows/atom-sglang-test.yaml	Updates PR CI accuracy threshold and adds DeepSeek-R1 FP4 TP4 to the matrix.
.github/workflows/atom-sglang-accuracy-validation.yaml	Adds manual toggle + nightly coverage for DeepSeek-R1 FP4 TP4; aligns FP8 TP4 threshold.
.github/benchmark/sglang_models_accuracy.json	Adds/updates dashboard metadata for the two DeepSeek-R1 TP4 accuracy entries (thresholds, baseline fields).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

[zhuyuhua-v]

still wip?

fixed in #747

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 6 comments.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 9 comments.