Skip to content

fix(core): correct turn-latency metrics and idle-window bg status visibility - #6292

Merged
bug-ops merged 1 commit into
mainfrom
fix/6279-tui-metrics-observability
Jul 14, 2026
Merged

fix(core): correct turn-latency metrics and idle-window bg status visibility#6292
bug-ops merged 1 commit into
mainfrom
fix/6279-tui-metrics-observability

Conversation

@bug-ops

@bug-ops bug-ops commented Jul 14, 2026

Copy link
Copy Markdown
Owner

Summary

  • MetricsBridge::WATCHED_SPANS watched the bare llm.chat span instead of llm.chat_with_tools, the span every tool-enabled turn (essentially all of them) actually dispatches through — the turn-latency panel showed llm:0ms even when the real LLM call took 25+ seconds. Auxiliary bare llm.chat calls in the same turn (self-check, compaction probe, etc.) were also silently overwriting the correct manually-tracked duration.
  • Fixed by watching llm.chat_with_tools, accumulating across a turn's multi-round tool loop (instead of overwriting), and scoping accumulation to spans nested under the main turn's llm.turn_call span so concurrent in-process sub-agent and scheduler chat_with_tools calls can't inflate or corrupt the main turn's llm_chat_ms (found during review).
  • The bg: N enrich, M telem TUI status segment only refreshed at the start of the next turn, so it stayed stale/invisible during the entire idle window after a response — exactly when background enrichment/telemetry extraction is actually running. Added a periodic BgMetricsTick to the existing Agent::next_event tokio::select! loop (no new tokio::spawn), using interval_at to defer the first tick and avoid racing the pre-existing channel-closed shutdown path (found during testing).

Closes #6275
Closes #6279

Test plan

  • cargo +nightly fmt --check
  • cargo clippy --profile ci --workspace --all-targets --features "desktop,ide,server,chat,pdf,scheduler,testing" -- -D warnings
  • cargo clippy --profile ci -p zeph-core --all-targets --features "profiling,scheduler" -- -D warnings (metrics_bridge.rs is profiling-gated, not covered by the CI feature matrix)
  • cargo nextest run --config-file .github/nextest.toml --workspace --features "desktop,ide,server,chat,pdf,scheduler" --lib --bins (13636 passed)
  • RUSTFLAGS="-D warnings" RUSTDOCFLAGS="--deny rustdoc::broken_intra_doc_links" cargo doc --no-deps --workspace --features "desktop,ide,server,chat,pdf,scheduler"
  • New/updated regression tests in metrics_bridge.rs, agent/utils.rs, agent/tests/bg_metrics_tick_race_tests.rs, agent/tests/agent_tests/lifecycle_tests.rs
  • Live --tui session confirmation per updated playbooks (metrics-bridge-watched-spans.md Scenario 2/2b/2c, bg-supervisor-phase2.md S9) — deferred to next live-testing cycle, coverage-status.md rows left Partial

@github-actions github-actions Bot added documentation Improvements or additions to documentation rust Rust code changes core zeph-core crate bug Something isn't working size/XL Extra large PR (500+ lines) labels Jul 14, 2026
@bug-ops
bug-ops enabled auto-merge (squash) July 14, 2026 18:53
…etrics tick

MetricsBridge::WATCHED_SPANS watched the bare llm.chat span, but every
tool-enabled turn dispatches through chat_with_tools() (llm.chat_with_tools)
instead, so the turn-latency panel showed llm:0ms even when the real call
took 25+ seconds. Auxiliary bare llm.chat calls in the same turn (self-check,
compaction probe, magic docs, etc.) were also silently overwriting the
correct manually-tracked duration. Now watches llm.chat_with_tools,
accumulates across a turn's multi-round tool loop, and scopes accumulation
to spans nested under the main turn's llm.turn_call span so concurrent
sub-agent and scheduler chat_with_tools calls can no longer inflate or
corrupt the main turn's llm_chat_ms.

The bg: N enrich, M telem status segment only refreshed at the start of the
next turn, so it stayed stale during the entire idle window after a
response while background enrichment/telemetry extraction was actually
running. Added a periodic BgMetricsTick to the existing Agent::next_event
select! loop that reuses reap_background_tasks_and_update_metrics between
turns, so the TUI reflects real in-flight background work continuously.
Uses interval_at to defer the first tick and avoid racing the pre-existing
channel-closed shutdown path.

Closes #6275
Closes #6279
@bug-ops
bug-ops force-pushed the fix/6279-tui-metrics-observability branch from b10617d to 61fd32d Compare July 14, 2026 18:56
@bug-ops
bug-ops merged commit 8584c13 into main Jul 14, 2026
43 checks passed
@bug-ops
bug-ops deleted the fix/6279-tui-metrics-observability branch July 14, 2026 19:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working core zeph-core crate documentation Improvements or additions to documentation rust Rust code changes size/XL Extra large PR (500+ lines)

Projects

None yet

1 participant