[AMD][MI355X] DSv4 SGLang agentic: v0.5.15 image, router, HiCache + MTP

[seungrokj]

Summary

• Refactor DSv4 FP4 MI355X SGLang agentic launcher (clean indentation, SGLANG_ROUTER_CMD pattern, env-var defaults)
• Rename config key dsv4-fp4-mi355x-sglang-agentic → dsv4-fp4-mi355x-sglang-agentic-hicache
• Bump image to lmsysorg/sglang-rocm:v0.5.15-rocm700-mi35x-20260713
• Add vllm-router 0.1.14 and mooncake 0.3.11.post1 KV offload backend to DPA sweep configs
• Exclude mia1-p01-g12 from agentic salloc
• Update perf-changelog entries

Test plan

• Full sweep validation on MI355X cluster
• Verify HiCache configs launch correctly
• Verify DPA + router configs launch correctly

🤖 Generated with Claude Code

[claude]

🔴 PARALLEL_ARGS is appended to at lines 95/104 (--dp/--enable-dp-attention/--enable-prefill-delayer/--enable-two-batch-overlap for DP_ATTENTION=true, and --ep-size for EP_SIZE>1), but line 119 does a plain reassignment PARALLEL_ARGS=(--tensor-parallel-size "$TP") that wipes out everything appended earlier. As a result, every dp-attn:true / ep-size>1 row in the new sweep (all the agentic-hicache DPA/HiCache rows) launches sglang as plain TP while the router is still started with --dp-aware/consistent_hashing, expecting DP ranks that no longer exist.

Extended reasoning...

The bug: In benchmarks/single_node/agentic/dsv4_fp4_mi355x_sglang.sh, PARALLEL_ARGS is built up incrementally with += in two places:

• Inside the if [ "$DP_ATTENTION" = "true" ] block: PARALLEL_ARGS+=(--dp "$TP" --enable-dp-attention --enable-prefill-delayer --enable-two-batch-overlap)
• Inside the if [ "${EP_SIZE:-1}" -gt 1 ] block: PARALLEL_ARGS+=(--ep-size "$EP_SIZE")

Both of these run before the array is initialized. Then, unconditionally, a later line does:

PARALLEL_ARGS=(--tensor-parallel-size "$TP")

This is a plain assignment, not +=. It discards every element that was appended above and replaces the array with just --tensor-parallel-size $TP. Since this line executes after both append blocks, PARALLEL_ARGS never contains --dp, --enable-dp-attention, --enable-prefill-delayer, --enable-two-batch-overlap, or --ep-size by the time it's expanded into SGLANG_CMD ("${PARALLEL_ARGS[@]}") and handed to sglang.launch_server.

Why nothing catches this: the script runs under set -euo pipefail, but appending to an unset array with += does not error under nounset — bash implicitly creates the array on first append — so the DP/EP blocks execute silently without any indication that their output is about to be thrown away.

Why this matters for this PR specifically: the diff that moved the base PARALLEL_ARGS=(--tensor-parallel-size "$TP") initialization is new — in the pre-PR version of this file, that same line existed before the DP-attention append block, so it worked correctly. The refactor relocated the PARALLEL_ARGS=(...) init to right before the SGLANG_CMD construction, after the DP/EP blocks, turning a correct pattern into a silent clobber.

Concrete walkthrough for one of the new sweep rows, e.g. { tp: 8, dp-attn: true, kv-offloading: none, conc-list: [16, 32, 48, 64] }:

1. DP_ATTENTION=true → PARALLEL_ARGS+=(--dp 8 --enable-dp-attention --enable-prefill-delayer --enable-two-batch-overlap). PARALLEL_ARGS is now (--dp 8 --enable-dp-attention --enable-prefill-delayer --enable-two-batch-overlap).
2. EP_SIZE is unset/1, so the EP block is skipped.
3. USE_SGLANG_ROUTER=true is set, SGLANG_BACKEND_PORT=$((PORT+1)), the router command is prepared.
4. Later: PARALLEL_ARGS=(--tensor-parallel-size "$TP") executes — PARALLEL_ARGS is now just (--tensor-parallel-size 8), and everything from step 1 is gone.
5. SGLANG_CMD is built with "${PARALLEL_ARGS[@]}" → the server launches as plain TP8, no DP-attention, no two-batch-overlap.
6. But USE_SGLANG_ROUTER=true still causes the router to start with --dp-aware and --policy consistent_hashing, routing to a backend that has zero DP ranks — a topology mismatch between what the router expects and what the server actually is.

Impact: every dp-attn:true row in the new dsv4-fp4-mi355x-sglang-agentic-hicache sweep (conc 16/32/48/64 for plain DPA, and 80/96 for the HiCache+DRAM-offload rows) silently degrades to plain TP8, defeating the entire point of the DPA/HiCache sweep added by this PR, while the router/backend pairing is left inconsistent.

Fix: either change the base-init line to PARALLEL_ARGS+=(--tensor-parallel-size "$TP"), or move the base initialization above the DP-attention/EP-size append blocks (restoring the original ordering).

[claude]

🟡 The new perf-changelog.yaml entry for dsv4-fp4-mi355x-sglang-agentic-hicache sets pr-link to pull/2146, but this changelog entry is actually introduced by this PR, #2199. Every other entry in the file links to the PR that introduced it, so this breaks the convention; please update the link to #2199 (and it looks like the description bullets were also copy-pasted from #2146 and don't match this diff, e.g. "disable-radix-cache" vs. the actual SGLANG_ENABLE_UNIFIED_RADIX_TREE=1 change, and "Sweep conc=48" vs. the actual conc-lists).

Extended reasoning...

What the bug is

The new perf-changelog.yaml entry for config-key dsv4-fp4-mi355x-sglang-agentic-hicache (added at the bottom of the file, lines 4753-4764) sets:

pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2146

But per the PR metadata, this changelog entry is being introduced by this PR, #2199. Every other entry in perf-changelog.yaml links to the PR that actually introduced it (the immediately preceding entries link to #2144, #2169, #2137, etc.), so the convention throughout the file is consistent, and pull/2146 breaks it.

Why this happened (copy-paste leftover)

This looks like the whole changelog block — link included — was copy-pasted from an earlier, unrelated PR (#2146) rather than authored fresh for #2199. The description bullets corroborate this: they describe things that do not match this PR's actual diff:

• The entry claims "disable-radix-cache", but the launcher (benchmarks/single_node/agentic/dsv4_fp4_mi355x_sglang.sh) never passes --disable-radix-cache; it instead exports SGLANG_ENABLE_UNIFIED_RADIX_TREE=1, which enables a unified radix tree — the opposite framing.
• The entry claims "Sweep conc=48 across TP8 +/- DPA +/- HiCache", but the actual configs/amd-master.yaml search-space for this key sweeps conc-list: [1, 2, 4, 8], [16, 32, 48, 64], and [80, 96] — conc=48 only appears inside the DPA-no-HiCache row, not as a dedicated cross-cutting sweep, and the HiCache row itself only covers [80, 96].

So the whole entry (link + several bullets) appears lifted wholesale from the earlier #2146 hicache work rather than describing this PR's actual changes.

Impact

This is a documentation/traceability field only — pr-link is not consumed by any benchmark execution path, so nothing breaks functionally, no runs fail, and no results are miscomputed. The impact is limited to anyone using the changelog to trace which PR introduced a given recipe change (e.g. for auditing or bisecting a perf regression) getting sent to the wrong PR (#2146 instead of #2199).

Step-by-step proof

1. Open perf-changelog.yaml lines 4753-4764 — the new entry for dsv4-fp4-mi355x-sglang-agentic-hicache.
2. Note pr-link: .../pull/2146.
3. Check the PR metadata for the current PR: it is [AMD][MI355X] DSv4 SGLang agentic: v0.5.15 image, router, HiCache + MTP  #2199, and the diff for this PR is exactly what adds this changelog block (git blame/git log -p on perf-changelog.yaml would show this hunk was introduced in [AMD][MI355X] DSv4 SGLang agentic: v0.5.15 image, router, HiCache + MTP  #2199, not [AMD][MI355X] DSv4 SGLang agentic: v0.5.15 image, router, HiCache #2146).
4. Compare adjacent entries in the same file (e.g. the entry just above, pr-link ...Add Qwen3.5 FP8 GB300 disaggregated multinode SGLang benchmarks via Dynamo / 新增 Qwen3.5 FP8 GB300 分离式多节点 SGLang（Dynamo）基准测试 #2137) — each of those links to the PR whose diff actually introduced that entry, confirming the file-wide convention.
5. Cross-check the entry's own description against this PR's diff: "disable-radix-cache" is absent from the launcher script's SGLANG_CMD array, and there is no standalone "conc=48" sweep in the config — both signs that the description (and by extension the pr-link) were copied from a different PR's changelog entry rather than written for [AMD][MI355X] DSv4 SGLang agentic: v0.5.15 image, router, HiCache + MTP  #2199.

How to fix

Change pr-link to https://github.com/SemiAnalysisAI/InferenceX/pull/2199, and while touching it, reconcile the description bullets to reflect what this PR actually changed (unified radix tree env var, fp8_e4m3 KV cache, SGLang router for DP-attention, SGLANG_CMD array pattern, capture_cache_metrics, and the real conc-list sweep), dropping the stale "disable-radix-cache" / "Sweep conc=48" bullets.