Skip to content

feat(model-info): add per-turn cache diagnostics - #372

Open
testikun wants to merge 3 commits into
openpi-dev:mainfrom
testikun:codex/issue-156-cache-diagnostics
Open

feat(model-info): add per-turn cache diagnostics#372
testikun wants to merge 3 commits into
openpi-dev:mainfrom
testikun:codex/issue-156-cache-diagnostics

Conversation

@testikun

@testikun testikun commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Problem

Issue #156 identified that OpenPI only exposed a cumulative cache percentage. A warm prefix becoming cold could not be located to a turn, while treating every cacheRead=0 as an invalidation would create false claims for first turns, tiny prefixes, and best-effort providers.

Value

Extensions can now consume a bounded, provider-aware observation for each assistant turn and correlate cache changes with local lifecycle boundaries without changing the model context or the existing footer metric.

Approach

  • Add a pure cache diagnostics tracker with explicit first-turn, cold, warm, partial-hit, miss-after-warm-prefix, and unknown outcomes.
  • Require a prior 2,048-token warm prefix and an explicit-prefix provider contract before reporting a miss; OpenAI/Google best-effort and unknown providers remain unknown.
  • Fingerprint the actual system prompt and selected tool surface at Pi's before_agent_start seam.
  • Record model, thinking, tool surface, system prompt, compaction, and branch boundaries only as correlations. Every record keeps verifiedCause: null.
  • Emit observations on a dedicated extension event channel; preserve cumulative cachePercent, TUI state, and model-visible context unchanged.

Related to #156. Does not close the Issue. Remaining #156 work: a provider usage-field matrix; TTL expiry as its own kind only with evidence; at least one real OpenPI / Bare Pi / OMP Session trace replay before any opt-in marker.

Validation

  • bun run check — passed.
  • bun run test — passed: Node 1248 passed / 1 skipped; Vitest 30 passed.
  • Node 22.19 focused tests — 11 passed.
  • git diff --check — passed.

No UI change was made, so a UI screenshot is not applicable.

Impact

  • User-visible behavior: none; no marker is enabled by default.
  • Model-visible context/tools: none.
  • Runtime/lifecycle: one bounded observation is emitted after each assistant turn; tracker state resets per Session.
  • Persisted config/data: none.
  • Compatibility/risk: additive event surface. Provider classification is intentionally conservative and unsupported providers remain unknown.

A late turn_end after /new, /reload, or quit could reuse a stale warm baseline and emit a false observation. Clear the tracker and pending identity at the same lifecycle boundary as session metrics.

@tt-a1i tt-a1i left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved at exact head 7e283300f61f1e3bd5aac4c934e2eef95912c4c4. I independently reviewed the scoped implementation and boundary conditions, verified the current diff against main, and found no unresolved P0/P1 issues. Required CI checks are green at this head.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants