Skip to content

[Bug]: 2026.6.9 re-busts prompt prefix caching — per-run :run:<UUID> injected into the system-prompt "Runtime:" line (regression of #43148) #96677

Closed

Description

@lujiaweichn

Summary

Since 2026.6.9 (still present in 2026.6.10, current latest), isolated cron runs embed a per-run UUID inside the session= field of the system prompt's Runtime: line. Because that UUID changes every run, byte-exact prefix caching is broken for everything after it — notably the ~8k-token tool catalog. Every cron LLM call then pays full uncached input. 2026.6.8 is unaffected. This re-introduces the class fixed in #43148 (related: #19989, #26750).

Affected versions

  • Introduced: 2026.6.9 · Confirmed still broken: 2026.6.10 (npm latest) · Last good: 2026.6.8

Mechanism

The Runtime: line changed from (6.8):

Runtime: agent=<agent> | session=agent:<agent>:cron:<jobid> | host=… | os=…

to (6.9 / 6.10):

Runtime: agent=<agent> | session=agent:<agent>:cron:<jobid>:run:<RUN_UUID> | host=… | os=…

<RUN_UUID> is unique per run → the cached prefix is no longer byte-stable → tool catalog etc. miss cache and are re-billed every call. Provider-agnostic.

Likely origin: #91685 ("refuse keyless implicit isolated cron delivery inherited from shared agent-main bucket", in 2026.6.9) gives each isolated cron run a distinct per-run session key, which is rendered verbatim into the cached system prompt.

Evidence

Same job, only the version changed:

version Runtime: has :run:<UUID> input_tokens / call
2026.6.8 no ~200 (cached)
2026.6.9 / 2026.6.10 yes ~11,500 (uncached)

Reproduced on two independent metered providers (both jumped identically → not provider-specific). For multi-iteration cron agents (10–25 calls/run, each re-sending growing context) this compounds to ~1.5–2M tokens/run — a ~10x cost increase on metered providers.

Suggested fix

Keep the volatile per-run id out of the cached system prefix:

  1. Don't render the cron delivery run-id into the system prompt at all (keep it internal to delivery routing); or
  2. Move the Runtime: line (or just its volatile fields) below the cache breakpoint / into a separate non-cached system message — the approach already discussed in [Bug]: Changing system prompt causes cache invalidations #43148; or
  3. Split static (agent/host/os/model + tool catalog, above the breakpoint) from volatile fields (session/run id, timestamps, below it).

Repro

  1. Create an isolated cron job (sessionTarget: isolated) on a metered provider that reports input_tokens.
  2. Run on 2026.6.8 → note input_tokens/call and that the session field is …:cron:<jobid> (no :run:).
  3. Upgrade to 2026.6.9/2026.6.10, run again → session field is now …:cron:<jobid>:run:<UUID> and input_tokens jumps ~50x.

Activity

  1. clawsweeper commented on Jun 25, 2026

    @clawsweeper
    Contributor

    Codex review: keeping this open for maintainer follow-up; there is still a little grit to resolve. Reviewed June 25, 2026, 2:00 AM ET / 06:00 UTC.

    Summary
    Keep open: current main and v2026.6.10 still pass a per-run isolated cron session key into embedded execution and render that key in the prompt Runtime line, so the reported cache-busting mechanism remains source-reproducible.

    Reproducibility: yes. at source level: current main constructs a :run: isolated-cron session key, passes it into embedded execution, and renders it in the Runtime prompt line. I did not independently reproduce the live provider token jump.

    Root-cause cluster
    Relationship: canonical
    Canonical: #96677
    Summary: This issue is the current canonical tracker for the specific isolated-cron Runtime-line :run: prompt-cache regression; older prompt-cache issues cover related but distinct mechanisms.

    Members:

    Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

    Next step
    A focused repair is plausible from the current source path, but the worker should keep the scope to prompt/cache-facing identity and treat live provider proof as validation rather than redesign.

    Review details

    Best possible solution:

    Keep the per-run cron identity internal for routing and cleanup, but render or route a stable cron prompt identity so repeated isolated runs keep a byte-identical cacheable prompt prefix.

    Do we have a high-confidence way to reproduce the issue?

    Yes at source level: current main constructs a :run: isolated-cron session key, passes it into embedded execution, and renders it in the Runtime prompt line. I did not independently reproduce the live provider token jump.

    Is this the best way to solve the issue?

    Yes, the requested repair direction is the right narrow fix: keep run identity internal and make prompt/cache-facing identity stable. Reusing one session across runs would weaken isolation, and adding a new config knob is not required for the core bug.

    AGENTS.md: found and applied where relevant.

    Remaining risk / open question:

    • I did not run a live metered-provider cache trace, so the exact token multiplier remains reporter-provided even though the volatile prompt field is source-reproducible.
    • A repair must keep per-run transcript, abort, cleanup, delivery, and subagent isolation intact while changing only prompt-facing or provider-cache-facing identity.

    Codex review notes: model internal, reasoning high; reviewed against 56d95b18f4bc.

    Label changes

    Label changes:

    • add issue-rating: 🦞 diamond lobster: Current issue advisory state selects this label.
    • add clawsweeper:source-repro: Current issue advisory state selects this label.
    • remove clawsweeper:needs-live-repro: Current issue advisory state no longer selects this label.
    • remove issue-rating: 🐚 platinum hermit: Current issue advisory state no longer selects this label.

    Label justifications:

    • P2: This is a source-backed prompt-cache cost regression with a bounded cron/provider surface and no outage, data loss, or security bypass.
    • impact:session-state: The issue is caused by per-run isolated cron session identity leaking into model-facing runtime prompt state.
    • impact:other: The concrete maintainer-visible impact is wasted model input tokens and degraded provider prompt-cache reuse.
    Evidence reviewed

    Acceptance criteria:

    • node scripts/run-vitest.mjs src/cron/isolated-agent/run.session-key-isolation.test.ts src/agents/system-prompt.test.ts src/agents/embedded-agent-runner/prompt-cache-observability.test.ts
    • node scripts/run-vitest.mjs src/llm/providers/anthropic.test.ts src/llm/providers/openai-completions.test.ts
    • If provider payload shaping changes, collect redacted live cache-read/cache-write proof for one isolated cron job before closing the issue.

    What I checked:

    Likely related people:

    • steipete: Commit 91f75a2 made isolated cron runs force fresh session IDs, which is the volatile identity family implicated here. (role: introduced isolated-cron freshness behavior; confidence: high; commits: 91f75a2b33b6; files: src/cron/isolated-agent/run.ts)
    • vincentkoc: Commits 64f2890 and 4798e12 introduced the cache-boundary and transport surfaces that any prompt-cache fix must preserve. (role: prompt-cache and transport area contributor; confidence: high; commits: 64f28906de09, 4798e125f40c; files: src/agents/system-prompt.ts, src/agents/system-prompt-cache-boundary.ts, src/agents/openai-transport-stream.ts)
    • nxmxbbd: The reporter linked the merged cron delivery PR, and d4833e2 recently touched cron delivery/session handling adjacent to this path. (role: recent adjacent cron delivery contributor; confidence: medium; commits: d4833e27c775; files: src/cron/isolated-agent/run.ts, src/cron/isolated-agent/delivery-target.ts)
    How this review workflow works
    • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
    • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
    • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
    • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
    • Maintainers can also comment @clawsweeper review to request a fresh review only.
    • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
    • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
    • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.
  2. added
    P2Normal backlog priority with limited blast radius.
    impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.
    impact:otherThis issue has meaningful maintainer-visible impact outside the owned taxonomy.
    issue-rating: 🐚 platinum hermitGood issue quality with a plausible reproduction path needing some confirmation.
    clawsweeper:needs-live-reproClawSweeper needs live local, crabbox, or manual validation to confirm this issue.
    issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.
    clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.
    and removed
    clawsweeper:needs-live-reproClawSweeper needs live local, crabbox, or manual validation to confirm this issue.
    issue-rating: 🐚 platinum hermitGood issue quality with a plausible reproduction path needing some confirmation.
    on Jun 25, 2026
  3. locked as resolved and limited conversation to collaborators on Aug 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Normal backlog priority with limited blast radius.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.impact:otherThis issue has meaningful maintainer-visible impact outside the owned taxonomy.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions