fix: remove metric publisher tight loop

[PeaBrane]

Overview:

as titled

Summary by CodeRabbit

• Bug Fixes
  • Improved stability of the metrics publishing system by reducing excessive re-publishing cycles. The system now implements a delayed retry mechanism to prevent rapid publish loops.

[coderabbitai]

Walkthrough

The timer reset logic in the KV router's metric publisher was modified to delay subsequent metric publication attempts. Instead of rearming the timer to 1ms on metric changes, it now resets to 3600 seconds, reducing rapid republication cycles.

Changes

Cohort / File(s)	Summary
Timer reset logic adjustment lib/llm/src/kv_router/publisher.rs	Changed background timer reset from 1ms to 3600 seconds after metrics publishing to prevent rapid re-publishing loops

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

• Review the timer state management and ensure the 3600-second interval aligns with intended publishing frequency
• Verify that metric change detection still triggers timely updates when actual changes occur
• Check for any edge cases where delayed publishing might conflict with critical metric reporting requirements

Poem

A rabbit hops through timers bright,
One ms? No—too quick, not right!
Now 3600 seconds we shall wait,
Less chaos means a calmer state. 🐰⏱️
No more loops that spin and race,
Just peaceful metrics, set their pace!

Pre-merge checks

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description Check	⚠️ Warning	The PR description is incomplete and does not follow the required template structure. The Overview section only contains "as titled" instead of providing a meaningful description of the pull request. The description is entirely missing the Details section (describing the changes made), the Where should the reviewer start section (identifying key files for review), and the Related Issues section (linking to any associated issues). This minimal description fails to provide reviewers with the necessary context to understand and evaluate the changes effectively.	The author should expand the PR description to follow the provided template. At minimum, the Details section should explain the specific changes to the timer reset logic (changing from 1ms to 3600 seconds), the Where should the reviewer start section should highlight lib/llm/src/kv_router/publisher.rs for review, and the Related Issues section should be populated if this PR closes or relates to any GitHub issues.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title Check	✅ Passed	The PR title "fix: remove metric publisher tight loop" directly aligns with the primary change in the pull request. The raw summary confirms that the change involves modifying the timer reset mechanism to prevent rapid re-publishing loops, which is exactly what the title communicates. The title is concise, specific, and clearly conveys the main objective of the fix without unnecessary details.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

---

📜 Recent review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e6809ff and ef0e661.

📒 Files selected for processing (1)

• lib/llm/src/kv_router/publisher.rs (1 hunks)

🧰 Additional context used

🧠 Learnings (2)

📓 Common learnings

Learnt from: PeaBrane
Repo: ai-dynamo/dynamo PR: 3095
File: lib/llm/src/kv_router/indexer.rs:0-0
Timestamp: 2025-09-17T20:55:06.333Z
Learning: When PeaBrane encounters a complex implementation issue that would significantly expand PR scope (like the remove_worker_sender method in lib/llm/src/kv_router/indexer.rs that required thread-safe map updates and proper shard targeting), they prefer to remove the problematic implementation entirely rather than rush a partial fix, deferring the proper solution to a future PR.

Learnt from: PeaBrane
Repo: ai-dynamo/dynamo PR: 1285
File: lib/llm/src/kv_router/scheduler.rs:260-266
Timestamp: 2025-05-30T06:34:12.785Z
Learning: In the KV router scheduler code, PeaBrane prefers fail-fast behavior over silent failure handling. When accessing worker metrics data that could be out-of-bounds (like dp_rank indexing), explicit panics are preferred over graceful degradation with continue statements to ensure data integrity issues are caught early.

Learnt from: PeaBrane
Repo: ai-dynamo/dynamo PR: 3184
File: docs/architecture/kv_cache_routing.md:70-73
Timestamp: 2025-09-23T20:08:37.105Z
Learning: PeaBrane prefers to keep documentation diagrams simplified to avoid visual overload, even when this means sacrificing some technical precision for the sake of clarity and comprehension. They prioritize pedagogical effectiveness over exhaustive technical detail in architectural diagrams.

Learnt from: PeaBrane
Repo: ai-dynamo/dynamo PR: 2756
File: lib/llm/src/kv_router/subscriber.rs:36-44
Timestamp: 2025-08-29T10:03:48.330Z
Learning: PeaBrane prefers to keep PRs contained in scope and is willing to defer technical improvements to future PRs when the current implementation works for the immediate use case. They acknowledge technical debt but prioritize deliverability over completeness in individual PRs.

Learnt from: PeaBrane
Repo: ai-dynamo/dynamo PR: 3095
File: lib/llm/src/kv_router/subscriber.rs:200-223
Timestamp: 2025-09-17T20:55:41.416Z
Learning: In the dynamo codebase, PeaBrane prefers to maintain consistency with existing etcd key parsing patterns (like splitting on '/' and parsing the last segment) rather than introducing more robust parsing approaches, even when the current approach might be brittle, to keep the codebase aligned and avoid divergent patterns.

📚 Learning: 2025-09-03T19:31:32.621Z

Learnt from: PeaBrane
Repo: ai-dynamo/dynamo PR: 2840
File: lib/llm/src/kv_router/sequence.rs:86-88
Timestamp: 2025-09-03T19:31:32.621Z
Learning: PeaBrane chose to defer fixing the corner case where a single late-arriving request might never expire in the ActiveSequences expiry mechanism (lib/llm/src/kv_router/sequence.rs). They prefer to avoid adding a background loop for periodic cleanup at this time, accepting the technical debt to keep the current PR scope contained.

Applied to files:

• lib/llm/src/kv_router/publisher.rs

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (16)

• GitHub Check: trtllm (amd64)
• GitHub Check: operator (amd64)
• GitHub Check: trtllm (arm64)
• GitHub Check: vllm (arm64)
• GitHub Check: vllm (amd64)
• GitHub Check: operator (arm64)
• GitHub Check: sglang (arm64)
• GitHub Check: sglang (amd64)
• GitHub Check: Build and Test - dynamo
• GitHub Check: clippy (launch/dynamo-run)
• GitHub Check: tests (lib/runtime/examples)
• GitHub Check: tests (lib/bindings/python)
• GitHub Check: tests (.)
• GitHub Check: clippy (.)
• GitHub Check: tests (launch/dynamo-run)
• GitHub Check: clippy (lib/bindings/python)

🔇 Additional comments (1)

lib/llm/src/kv_router/publisher.rs (1)

863-868: LGTM! Good fix for the tight loop issue.

The timer reset to 3600 seconds effectively "parks" the timer until the next metric change triggers a reset to 1ms (line 844-846). This prevents the timer from firing repeatedly when there are no pending metrics to publish, solving the tight loop problem described in the PR title.

The logic is correct and the comment clearly explains the intent.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}