Reliable semantic context compaction - #67
Merged
Merged
Conversation
Pinned <knowledge_memory> retention used fixed per-field caps tuned for a 100k semantic target, so large windows threw away recoverable evidence they could afford to keep while small BYOK windows let the verbatim, cutoff-exempt block crowd out the live working set. Per-field caps now scale by clamp(targetTokens / 100_000, 0.5, 3.0) and the whole block is bounded by max(1_500, floor(target * 0.25)) estimated tokens with oldest-first eviction in a fixed field order; Goal and Next Action truncate toward 480/240-character floors instead of being dropped. Three baseline caps are raised (decisions 8->12, blockers 8->12, next action 1,000->1,400 chars) and near-duplicate entries are deduped by normalized text so repeats stop consuming budget. Compaction was also invisible until it finished, so a long pass looked like a hang. An additive context_compaction_status print-mode event (state 'started' | 'settled', with required runId/ancestorRunIds) drives a live 'Compacting context...' card and status chip that settles in place into the result card and terminates as 'Compaction interrupted' on abort. State is keyed by run, never by the forwarding-rewritten agentId, so subagent passes never render as root-level state or cross-settle the root run. Docs and CHANGELOGs describe the same contract.
Semantic compaction only ever happened when an agent had a handleSteps generator that spawned the context-pruner itself, so a prompt-only template had nothing between it and the mechanical trim: it silently lost the oldest turns instead of getting an evidence-preserving pass. The runtime now drives one pruner pass itself for those templates through packages/agent-runtime/src/util/runtime-semantic-compaction.ts, gated by the same spawnableAgents permission contract as spawn_agent_inline, so a consumer agent that does not declare context-pruner pays for no extra child run and never has its transcript rewritten. Repeated passes that reclaim nothing were previously paid for every iteration. A new transient AgentState.suppressSemanticCompaction advisory records that consecutive announced passes measurably failed to shrink history, after which both pruner spawn paths stop spawning for the rest of the turn. It is loop-owned and reset at loop entry, so a persisted or inherited true can never disable compaction for a later recoverable run, and it lowers no budget and drops no pinned state. Pruner identity is now matched by bare agent id via isContextPrunerAgentId in the inline spawn, the runtime-driven pass, and the run-programmatic-step pruner param injection. A publisher-qualified or version-pinned pruner previously missed the injected semanticBudget, taskMemory, workspaceState, and clamped maxContextLength, fell back to its embedded budget arithmetic, and published a -1 task-memory revision, so commitTaskMemory rejected its set_messages and the announced compaction silently did not happen. The additive context_compaction_status print-mode event now requires runId, so a settled event can only clear its own run's live card, and the CLI renders a replayed pending card as interrupted rather than as a permanently compacting one. On the SDK side the request-time emergency trim subtracts this request's counted system-prompt and tool-schema tokens from the resolved limit and reports the message-only budget in triggerBudgetTokens and targetBudgetTokens, so those fields name the threshold actually applied to messages. Tool schemas are counted from a JSON-safe projection because published param types accept opaque Zod schemas. Covered by runtime loop tests, SDK context-window tests, and a new deterministic evals/compaction-retention scenario; docs/agents-and-tools.md and sdk/CHANGELOG.md record the widened pruner match and the trim budget change as consumer-visible.
Commit 1a4fb41 touched cli/src and common/src, so guard:memory-drift reported both knowledge files as stale against their sibling sources. Document the live context_compaction_status rendering with run-correlated pending/settled pairing on the CLI side, and the additive print-mode variant plus the transient loop-owned AgentState.suppressSemanticCompaction advisory with bare-agent-id pruner matching on the common side. Restores the printModeToolCallSchema bullet alongside the new entries.
The request-time trim charged every opaque Zod/Standard Schema inputSchema a flat 120 tokens, which materially under-reserved a large tool surface and made the brake believe more message budget was available than really was. Convert the schema with the AI SDK's asSchema, count that JSON Schema projection, and clamp it between a 120-token floor and an 8,000-token per-tool ceiling so the reservation tracks real size without one pathological schema collapsing the budget; conversion failures still fall back to the floor instead of throwing. Also address two reviewer advisories with no behavior change: move isContextPrunerAgentId and CONTEXT_PRUNER_AGENT_ID into a leaf context-pruner-identity module so identity consumers stop importing the semantic-compaction spawn path, and narrow runRuntimeSemanticCompaction to Promise<void>, dropping its unreachable suppressSemanticCompaction re-check now that run-agent-step.ts is documented as owning that anti-thrash gate.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Makes semantic context compaction work reliably for every agent shape, and stops the SDK request-time emergency brake from under-reserving its non-message overhead.
Runtime
Live status
The additive context_compaction_status print-mode event pairs started and settled by a required runId, so nested agent loops cannot cross-settle each other; suppressed iterations emit neither half. In the CLI a replayed pending card renders as interrupted rather than a permanently spinning Compacting context card.
SDK
getMessagesForModelContext now subtracts the counted system-prompt and tool-schema tokens from the message limit. Opaque Zod or Standard Schema inputSchema values are converted to real JSON Schema and counted from that projection, clamped between a 120-token floor and an 8000-token per-tool ceiling, never throwing on a cycle. triggerBudgetTokens and targetBudgetTokens now report the threshold actually applied to messages.
Testing
New coverage includes the runtime-driven pass, the permission gate, pinned-pruner spawn and param injection, suppression and its loop-entry reset, the announce and settle invariant under failure, and the SDK schema-counting floor, ceiling and parity cases.
Consumer notes
The SDK changelog documents each consumer-visible change: the additive context_compaction_status event, the additive AgentState.suppressSemanticCompaction field, the pinned-pruner widening (with a rename migration if an unrelated agent was published under the bare id context-pruner), and the tightened trim budget including the trigger and target budget semantics change.
This change is