Skip to content

feat(core): diagnose prompt cache prefix changes - #39139

Merged
kitlangton merged 5 commits into
v2from
cache-prefix-diagnostics
Jul 27, 2026
Merged

feat(core): diagnose prompt cache prefix changes#39139
kitlangton merged 5 commits into
v2from
cache-prefix-diagnostics

Conversation

@kitlangton

@kitlangton kitlangton commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

What

Add opt-in prompt cache diagnostics for V2 model requests. Set:

OPENCODE_PROMPT_CACHE_DIAGNOSTICS=1

Consecutive provider-bound Physical Attempts in a Session are then classified as:

  • initial
  • stable
  • append-only
  • changed, with the first changed component identified as settings, tools, system, or messages

This makes intermittent provider cache misses actionable without enabling unrelated debug logs or logging prompt, tool, or message contents.

How

  • packages/core/src/session/prompt-cache-diagnostics.ts builds truncated SHA-256 snapshots of cache-sensitive request components and compares their ordered prefixes.
  • V2 SessionModelRequest allocates diagnostics state and hashes requests only when OPENCODE_PROMPT_CACHE_DIAGNOSTICS is enabled.
  • Enabled diagnostics emit structured info logs, so the dedicated flag is sufficient on its own.
  • V2 compaction runs before SessionModelRequest.prepare, so snapshots correspond to provider-bound Physical Attempts.
  • The Location-scoped in-memory snapshot map is capped at 100 Sessions using insertion order as an LRU approximation.
  • Runner coverage verifies that a real instruction update preserves settings, tools, system, and prior history while appending chronological messages.

Example structured log classifications:

prompt cache prefix status=append-only previousMessages=1 currentMessages=3
prompt cache prefix status=changed component=tools index=4 label=execute

Scope

  • V2 only: this changes packages/core; it does not touch V1 packages/opencode.
  • Does not alter requests, tool ordering, cache keys, cache breakpoints, or provider behavior.
  • Does not log raw prompts, hashes, tool schemas, system text, or message text.
  • Does no snapshot allocation or hashing when the dedicated flag is unset.
  • Does not claim that a stable logical request guarantees a provider cache hit; provider-side routing, eviction, thresholds, and accounting remain separate concerns.
  • Does not fingerprint the final serialized HTTP body. External wire capture remains the deeper fallback when the logical prefix is stable.

Testing

  • bun run test test/prompt-cache-diagnostics.test.ts from packages/core: 5 passed
  • bun typecheck from packages/core
  • Push hook: bun turbo typecheck --concurrency=3: 32 packages passed
  • git diff --check
  • The current v2 base independently fails existing session-runner.test.ts expectations after recent Code Mode/tool-result changes; reproduced from an untouched origin/v2 worktree. The added prefix assertion passes before the existing stale expectation fails.

@kitlangton
kitlangton force-pushed the cache-prefix-diagnostics branch from f19144e to 1e479c4 Compare July 27, 2026 16:21
@kitlangton
kitlangton changed the base branch from dev to v2 July 27, 2026 16:22
@kitlangton
kitlangton force-pushed the cache-prefix-diagnostics branch from 0778f17 to 9ccff5d Compare July 27, 2026 17:43
@kitlangton
kitlangton merged commit 7eb51d0 into v2 Jul 27, 2026
5 of 7 checks passed
@kitlangton
kitlangton deleted the cache-prefix-diagnostics branch July 27, 2026 17:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant