Auto-compact idle threads before the prompt cache expires

[josephv123]

Problem

When I step away from a large thread, its prompt cache expires. The cache lasts 60 minutes on a Claude subscription and 5 minutes on an API key or in overage. After that, my next message re-reads the whole context at the uncached rate, and so does compacting it. The "Resume with less context" banner only shows up after 70 idle minutes, so by the time it appears the cache has already expired and compacting is the expensive option.

Proposal

Add an opt-in setting that compacts a thread automatically after a period of inactivity, timed to happen while the cache is still warm.

• Setting: "Auto-compact idle threads". Off by default, with an idle delay (default a few minutes before the cache TTL, e.g. ~50 min for a 1-hour cache) and a minimum context size (e.g. 100K tokens) so small threads are left alone.
• Runs on the server, so it works without a connected client. The app is usually closed or in the background when this matters.
• Skips busy threads: anything with an active turn, a pending approval or input request, or live background work.
• Leaves a visible marker in the thread (e.g. "Auto-compacted after 55 min idle") so it's clear what happened.
• Covers Codex too, through the existing compactThread.

Why

The measurements in #13988 show the gap. Compacting a ~210K-token thread cost about $0.10 while the cache was warm and about $0.90 once it was cold. Continuing the cold thread without compacting cost $1.67 for the first message. For subscription users, that difference counts directly against usage limits.

Related

• [Bug]: "Resume with less context" is offered only after the prompt cache has expired, so it always takes the expensive path #13988: the triage there identified compacting or notifying before the cache window closes as the actionable part
• feat(claude): compact old threads before they burn through usage #8144: the current 70-minute banner
• Add a "Compact now" action to the context window meter #8100: "Compact now" action
• Sidebar: show each thread's remaining prompt-cache window #7590: showing each thread's remaining cache window
• [Bug]: Idle session teardown can trigger full Claude prompt-cache rebuilds after repository changes #10600: idle session teardown and cache rebuilds

[teoobarca]

+1, and one finding that makes this T3's job rather than Claude Code's:

Claude Code 2.1.286+ now does this natively ("Compacted while idle, before the prompt cache expired"): at ~90 % of the 1 h TTL (~54 min idle), context ≥ 200k (CLAUDE_CODE_IDLE_COMPACT_MIN_TOKENS, floor 100k), opt-out idleCompaction: false; see anthropics/claude-code#98747. It never fires in T3, for two independent reasons:

1. Its timer only arms for an interactive UI or an allow-listed entrypoint (claude-vscode, claude-desktop, remote*, …). T3 spawns Claude through the Agent SDK as sdk-ts.
2. Even with a different entrypoint, ProviderSessionManager.ts:54 releases an idle Claude process after 30 min (DEFAULT_IDLE_TIMEOUT_MS), well before minute ~54.

So it has to be a server-side sweep in T3, and most pieces already exist (commit cafe620e):

• tokenUsage.promptCacheTtlMs is already derived from ephemeral_1h / ephemeral_5m cache writes and persisted per turn (orchestrationV2.ts:1066).
• UsageLimitRecoveryWorker is a close template: a scheduler sweep that queries candidates from the projections and dispatches a server-created message with an idempotent commandId, gated by a setting.
• Candidates: Claude, not running, nothing queued, no pending request, no pending background work, not archived, latest usedTokens ≥ threshold, 1 h TTL only, idle between 0.9×TTL and TTL, last run not already a /compact. Dispatch /compact in queue mode with commandId = idle-compact:<thread>:<providerTurnId> so each idle gap fires at most once.

Rough size: 250–400 LOC across 6–8 files including tests.

Open risk worth measuring first: the process is gone by then, so the compaction runs on a resumed process. It only hits the warm cache if the resumed request prefix (system prompt, tools, MCP servers) is byte-identical to the pre-idle one. If it isn't, this pays the cold read anyway. Keeping Claude processes alive past ~55 min would sidestep that at the cost of memory.

Since Claude Code users pushed back on silent idle compaction (anthropics/claude-code#98747), keeping it opt-in with a visible "auto-compacted after N min idle" marker, as proposed above, seems right.

[bman654]

"idle" should include "waiting" states (waiting on PR watch, waiting on background shell, waiting on subagents)...any state where the agent thread itself is doing nothing even if subagents or shells are running since they all pay the cache hit price when the background task eventually notifies. Extra points if the compact request can pass along instructions to remember the list of background tasks/waits/watches it has active