Skip to content

[None][perf] Skip transceiver tp_allgather when no sessions ever opened#14042

Open
Shixiaowei02 wants to merge 1 commit into
NVIDIA:feat/deepseek_v4from
Shixiaowei02:user/xiaoweis/skip-allgather
Open

[None][perf] Skip transceiver tp_allgather when no sessions ever opened#14042
Shixiaowei02 wants to merge 1 commit into
NVIDIA:feat/deepseek_v4from
Shixiaowei02:user/xiaoweis/skip-allgather

Conversation

@Shixiaowei02
Copy link
Copy Markdown
Collaborator

@Shixiaowei02 Shixiaowei02 commented May 12, 2026

@coderabbitai summary

Description

V4-Pro 8k/1k MTP3 disagg (ctx1 TP=4 + gen4 TP=8), concurrency=4/server (total 16), 128 prompts × 8 rounds, gb300:

Metric Before After Δ
output_throughput (tok/s) 600.1 626.5 +4.41%
user_throughput (tok/s) 45.87 47.38 +3.28%
mean_tpot_ms 6.373 6.126 −3.87%
mean_ttft_ms 16491 16300 −1.16%
mean_itl_ms 1434.9 1416.4 −1.29%
mean_e2el_ms 22354 21932 −1.89%
avg_decoded_tokens_per_iter 2.654 2.676 +0.83%
duration (s) 196.2 188.0 −4.22%

GPU compute is unchanged . The full 4.4% e2e throughput win comes from eliminating one tp_allgather per decode iter on the GEN worker, which had previously synced all 8 TP ranks every iter against an empty consensus list.

Test Coverage

PR Checklist

Please review the following before submitting your PR:

  • PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

  • PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

  • Test cases are provided for new code paths (see test instructions)

  • Any new dependencies have been scanned for license and vulnerabilities

  • CODEOWNERS updated if ownership changes

  • Documentation updated as needed

  • Update tava architecture diagram if there is a significant design change in PR.

  • The reviewers assigned automatically/manually are appropriate for the PR.

  • Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

@Shixiaowei02 Shixiaowei02 force-pushed the user/xiaoweis/skip-allgather branch from 64791f5 to 684e049 Compare May 12, 2026 08:21
@Shixiaowei02 Shixiaowei02 marked this pull request as ready for review May 12, 2026 08:21
@Shixiaowei02 Shixiaowei02 requested a review from a team as a code owner May 12, 2026 08:21
@Shixiaowei02 Shixiaowei02 requested review from chuangz0, liji-nv and suyoggupta and removed request for a team May 12, 2026 08:21
@Shixiaowei02 Shixiaowei02 requested a review from qiaoxj07 May 12, 2026 08:22
Signed-off-by: Xiaowei Shi <39303645+Shixiaowei02@users.noreply.github.com>
@Shixiaowei02 Shixiaowei02 force-pushed the user/xiaoweis/skip-allgather branch from 684e049 to 70162aa Compare May 12, 2026 13:30
@Shixiaowei02
Copy link
Copy Markdown
Collaborator Author

/bot run --add-multi-gpu-test --disable-fail-fast

@tensorrt-cicd
Copy link
Copy Markdown
Collaborator

PR_Github #47973 [ run ] triggered by Bot. Commit: 70162aa Link to invocation

@tensorrt-cicd
Copy link
Copy Markdown
Collaborator

PR_Github #47973 [ run ] completed with state SUCCESS. Commit: 70162aa
/LLM/main/L0_MergeRequest_PR pipeline #37813 completed with status: 'SUCCESS'

CI Report

Link to invocation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants