Skip to content

feat(ai): prompt-cache correctness fixes for Kimi, OpenRouter, and Bedrock - #766

Merged
code-yeongyu merged 7 commits into
mainfrom
code-yeongyu/cache-hit-ai-fixes
Aug 9, 2026
Merged

feat(ai): prompt-cache correctness fixes for Kimi, OpenRouter, and Bedrock#766
code-yeongyu merged 7 commits into
mainfrom
code-yeongyu/cache-hit-ai-fixes

Conversation

@code-yeongyu

@code-yeongyu code-yeongyu commented Aug 9, 2026

Copy link
Copy Markdown
Owner

Summary

This PR implements PR-A from the approved cache-hit maximization plan:

  • Approved plan: cache-hit-maximization.md
  • Scope: T1, T5, T14, T2, T3, T4, and T15 only.

Behavior changes

  • Parse Kimi's flat usage.cached_tokens as cache-read usage without changing existing precedence.
  • Use one shared Bedrock capability helper so only Claude Opus/Sonnet/Haiku 4.5 receive a one-hour cache TTL; resolver estimates now match the wire TTL.
  • Report a 300-second prompt-cache TTL for the actual Claude SDK OAuth model shape.
  • Add supportsPromptCacheKey compatibility detection and send stable, clamped prompt_cache_key values for Moonshot/Kimi requests.
  • Default OpenRouter session affinity and send the session ID in both x-session-id and body session_id.
  • Apply explicit OpenRouter cache markers only to anthropic/, qwen/, and google/ model prefixes after stripping one optional leading ~; runtime and generator logic are identical.
  • Regenerate and validate the Moonshot/OpenRouter catalog metadata and add package changelog/source change notes.

TDD evidence

Each implementation todo has captured failing and passing output under:

  • .omo/evidence/cache-hit-maximization/task-1-{red,green}.txt
  • .omo/evidence/cache-hit-maximization/task-5-{red,green}.txt
  • .omo/evidence/cache-hit-maximization/task-14-{red,green}.txt
  • .omo/evidence/cache-hit-maximization/task-2-{red,green}.txt
  • .omo/evidence/cache-hit-maximization/task-3-{red,green}.txt
  • .omo/evidence/cache-hit-maximization/task-4-{red,green}.txt
  • .omo/evidence/cache-hit-maximization/task-15-{red,green}.txt
  • T4 hydration receipts: .omo/evidence/cache-hit-maximization/task-4-hydrate.txt and task-4-model-data-check.txt
  • T15 prose receipt: .omo/evidence/cache-hit-maximization/task-15-changes.md.txt

These evidence files are task-local and intentionally ignored by git.

Verification

  • cd packages/ai && npm test -- test/ - 191 files passed, 1816 tests passed, 25 files / 806 tests skipped by existing suite conditions.
  • npm run check - passed.
  • npm run build - passed (required to materialize workspace package artifacts before the full AI suite).
  • npm run check:model-data - passed.
  • node scripts/check-pr-changelog.mjs --base origin/main - passed.
  • git diff --check origin/main...HEAD - passed.
  • Runtime and generator OpenRouter cache-control blocks were compared byte-for-byte and are identical.

Summary by cubic

Improves prompt cache correctness across OpenAI-compatible and Bedrock lanes to boost cache hits and accurate usage/TTL reporting. Aligns Kimi usage accounting, OpenRouter cache-control and affinity, and Bedrock TTL behavior.

  • New Features

    • Added supportsPromptCacheKey; sends a clamped prompt_cache_key for Moonshot/Kimi when caching is on.
    • Defaulted OpenRouter session affinity on; sends the same session ID in x-session-id and body session_id.
    • Applied explicit OpenRouter cache markers to anthropic/, qwen/, and google/ prefixes (handles one leading ~).
    • Regenerated and validated Moonshot/OpenRouter catalog metadata.
  • Bug Fixes

    • Parsed Kimi’s flat usage.cached_tokens as cache-read tokens without changing precedence.
    • Restricted Bedrock 1h prompt-cache TTL to Claude Opus/Sonnet/Haiku 4.5 only; others use 5m so resolver matches the wire TTL.
    • Reported a 300s prompt-cache TTL for the actual claude-sdk-oauth model shape.

Written for commit 1014c26. Summary will update on new commits.

Review in cubic

@code-yeongyu
code-yeongyu force-pushed the code-yeongyu/cache-hit-ai-fixes branch from 101f55c to 1014c26 Compare August 9, 2026 08:43
@code-yeongyu
code-yeongyu merged commit 19151c1 into main Aug 9, 2026
17 checks passed
@code-yeongyu
code-yeongyu deleted the code-yeongyu/cache-hit-ai-fixes branch August 9, 2026 08:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant