Refresh DeepSeek V4 GB300 TRT-LLM AgentX metrics by cquil11 · Pull Request #2774 · SemiAnalysisAI/InferenceX

cquil11 · 2026-08-28T04:57:33Z

Summary

Refresh DeepSeek V4 GB300 AgentX to collect TensorRT-LLM Prometheus metrics.
Use the pinned Dynamo fork for metrics collection with KV-event publishing disabled.
Preserve the existing TensorRT-LLM image and serving settings; validate KV-cache utilization in the collected metrics.

Upstream feature request: ai-dynamo/dynamo#14121

Note

Low Risk
Benchmark and changelog YAML only; no application or serving code paths change.

Overview
Refreshes six DeepSeek V4 Pro GB300 disaggregated Dynamo+TensorRT-LLM AgentX Slurm recipes so reruns can collect TensorRT-LLM server metrics (including KV-cache utilization), aligned with the prior GLM AgentX metrics refresh.

Dynamo is pinned by git hash (2cbbdc86…) instead of the dev wheel / identity.frameworks.dynamo version. Prefill and decode both set enable_iter_perf_stats: true. Frontend and benchmark env move KV_OFFLOADING from none to dram. The agentic client adds AIPERF_REQUIRED_SERVER_METRIC_PREFIX: trtllm_kv_cache_utilization so AIPerf fails fast if expected Prometheus metrics are missing. perf-changelog.yaml records the dsv4-fp4-gb300-dynamo-trt-agentx agentic-coding refresh for this PR.

^{Reviewed by Cursor Bugbot for commit a00b074. Bugbot is set up for automated code reviews on this repo. Configure here.}

github-actions · 2026-08-28T04:57:41Z

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

claude · 2026-08-28T05:10:21Z

@@ -88,7 +88,8 @@ backend:
      moe_expert_parallel_size: 8


🔴 Recipe modification (metrics/telemetry toggles across 6 GB300 DSv4 recipes) is missing the required perf-changelog.yaml entry mandated by AGENTS.md's non-negotiable invariant. [also at: perf-changelog.yaml:1 - Missing required perf-changelog.yaml entry for this recipe-modifying PR, violating AGENTS.md's non-negotiable benchmark…]

Extended reasoning...

AGENTS.md states every recipe addition or modification requires a new perf-changelog.yaml entry, and CONTRIBUTING.md restates this as a MUST; git diff on perf-changelog.yaml is empty for this change. After merge, these 6 recipe files diverge from their changelog history with no traceable rationale for the return_perf_metrics/enable_iter_perf_stats/publish_events_and_metrics/AIPERF_REQUIRED_SERVER_METRIC_PREFIX changes, breaking the append-only audit trail reviewers rely on to correlate recipe behavior changes with PRs.

Verification: Severity: normal (this PR triggers a non-negotiable, merge-gating invariant that it fails to satisfy). The diff (git diff 0ec742a2ba45a0cbc3be183b7697ebb90f35439b HEAD --stat) modifies exactly 6 recipe YAMLs under benchmarks/multi_node/srt-slurm-recipes/trtllm/deepseek-v4/agentx-gb300-20260811/ (36 insertions, 18 deletions) and touches nothing else. Confirmed the specific edits at 1p1d line…

github-actions · 2026-08-28T08:45:33Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33144826062
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=33144826062

github-actions · 2026-08-28T11:48:37Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33156571001
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=33156571001

cursor

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

There are 3 total unresolved issues (including 2 from previous reviews).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 2986619. Configure here.}

github-actions · 2026-08-28T22:02:30Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33168529787
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=33168529787

github-actions · 2026-08-28T22:59:01Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33215261579
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=33215261579

github-actions · 2026-08-28T23:16:45Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33218780724
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=33218780724

github-actions · 2026-08-29T23:45:59Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33219706423
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=33219706423

github-actions · 2026-08-31T17:06:03Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33219706423
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=33219706423

github-actions · 2026-09-01T06:04:59Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33418433573
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=33418433573

github-actions · 2026-09-01T09:27:50Z

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33418433573
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=33418433573

github-actions · 2026-09-01T13:14:26Z

@cquil11 staging run 33418433573. InferenceX-app staging workflow is now running. Existing staged runs will be preserved; this comment will be updated when staging completes.

github-actions · 2026-09-01T13:22:18Z

@cquil11 staged run 33418433573: https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-08-31~r33418433573

This run remains available across future /stage-results requests. Staging the same run ID again updates its staged data. Staging workflow

cquil11 · 2026-09-01T14:56:13Z

/stage-results 33418433573

cquil11 · 2026-09-01T14:56:20Z

/reuse-sweep-run 33418433573

github-actions · 2026-09-01T14:56:28Z

@cquil11 staging run 33418433573. InferenceX-app staging workflow is now running. Existing staged runs will be preserved; this comment will be updated when staging completes.

cquil11 · 2026-09-01T17:59:42Z

/reuse-sweep-run 33418433573

cquil11 · 2026-09-01T21:32:35Z

/reuse-sweep-run

# Conflicts: # perf-changelog.yaml

cquil11 · 2026-09-01T21:32:55Z

/reuse-sweep-run 33418433573

main appended five entries (#2770-#2774, #2790) at the end of the changelog while this branch appended the DSv4 MI355X entry, so the two collided on the same trailing lines. All are kept, upstream's first. Co-authored-by: Cursor <cursoragent@cursor.com>

cquil11 requested a review from a team August 28, 2026 04:57

github-project-automation Bot added this to InferenceMAX Board Aug 28, 2026

claude Bot reviewed Aug 28, 2026

View reviewed changes

cquil11 added the full-sweep-enabled label Aug 28, 2026

cursor Bot reviewed Aug 28, 2026

View reviewed changes

Comment thread utils/agentic/aggregation/backends/trtllm.py

Comment thread utils/agentic/aggregation/backends/trtllm.py

cursor Bot reviewed Aug 28, 2026

View reviewed changes

Comment thread benchmarks/multi_node/agentic_srt.sh

cquil11 added 9 commits August 31, 2026 12:12

config(dsv4): enable GB300 TRT-LLM metrics

f161a61

chore: add DeepSeek V4 metrics refresh changelog

b208a43

fix(dsv4): require iteration metrics

71bd965

fix(agentx): aggregate TensorRT-LLM metrics

96c5ef7

fix(agentic): target the SRT frontend host

9aae941

fix(trtllm): decouple metrics from KV events

ba1885a

Avoid detailed TRT metrics overhead

c6bb645

Use metrics-only Dynamo publisher

65a4743

fix: align DSv4 AgentX offload metadata

4468ccc

cquil11 force-pushed the agent/refresh-dsv4-gb300-trt-metrics branch from 4473e70 to 4468ccc Compare August 31, 2026 17:13

cquil11 force-pushed the agent/refresh-dsv4-gb300-trt-metrics branch from ade38fc to 4468ccc Compare September 1, 2026 20:27

docs: consolidate metrics refresh changelog [skip-sweep]

7b53505

Merge remote-tracking branch 'origin/main' into pr-2774-reuse-99580

a00b074

# Conflicts: # perf-changelog.yaml

cquil11 merged commit 5ecf6fd into main Sep 1, 2026
29 checks passed

cquil11 deleted the agent/refresh-dsv4-gb300-trt-metrics branch September 1, 2026 21:35

github-project-automation Bot moved this to Done in InferenceMAX Board Sep 1, 2026

Uh oh!

Conversation

cquil11 commented Aug 28, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

github-actions Bot commented Aug 28, 2026

Uh oh!

claude Bot Aug 28, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Aug 28, 2026

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Aug 28, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions Bot commented Aug 28, 2026

Uh oh!

github-actions Bot commented Aug 28, 2026

Uh oh!

github-actions Bot commented Aug 28, 2026

Uh oh!

github-actions Bot commented Aug 29, 2026

Uh oh!

github-actions Bot commented Aug 31, 2026

Uh oh!

github-actions Bot commented Sep 1, 2026

Uh oh!

github-actions Bot commented Sep 1, 2026

Uh oh!

github-actions Bot commented Sep 1, 2026

Uh oh!

github-actions Bot commented Sep 1, 2026

Uh oh!

cquil11 commented Sep 1, 2026

Uh oh!

cquil11 commented Sep 1, 2026

Uh oh!

github-actions Bot commented Sep 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cquil11 commented Sep 1, 2026

Uh oh!

cquil11 commented Sep 1, 2026

Uh oh!

cquil11 commented Sep 1, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

cquil11 commented Aug 28, 2026 •

edited by cursor Bot

Loading

github-actions Bot commented Sep 1, 2026 •

edited

Loading