fix(#288): cap sub-agent entry body retention (88% of heap) + dominator --owners analysis#320
Merged
Merged
Conversation
…alysis A dominator-tree heap analysis proved 7 live SubAgentWatcher instances retained 263 MB = 88% of the entire reachable main-process heap, via SubAgentWatcher.entriesByAgent (Map<agentId, RawEntry[]>) → entry → message.content → huge string. PR #300 capped the entry COUNT (MAX_RETAINED_ENTRIES_PER_AGENT=500) but NOT the per-entry BYTE SIZE, so each retained entry still pinned its full body (Read outputs, tool_result bodies, the skills-list attachment, [Truncated: PARTIAL view] reminders). 500 entries × multi-MB bodies × several agents × 7 watchers = 263 MB, and churning those strings every 600 ms poll fragmented V8 into ~1.5 GB RSS. PART 1 — truncate-at-retention (zero visible change): - SubAgentWatcher: new truncateEntryBodies() clamps every large free-text field (text/thinking text, tool_result content/output, large tool_use input string values, skill attachment.content, system-reminder text) to 8 KB BEFORE pushing into entriesByAgent, preserving all structural fields buildSubAgentState reads. 8 KB is ~100× the 80-char headline the renderer shows, so no rendered value can be clipped; the durable full transcript stays on disk. - codexSubagentState: dropped childEntriesByAgentId entirely. PR #317 kept a bounded 500-entry raw tail "for a future mini-feed" that nothing read — write-only dead weight pinning full bodies. The derived SubAgentState in childStateByAgentId already carries everything emit() ships, so removing the tail is invisible. The #317 full-read derivation logic is untouched. PART 2 — the tool that proved it: - analyze-heapsnapshot.mjs: new --owners mode builds the dominator tree (Cooper-Harvey-Kennedy), computes retained size per node, and reports the top individual retainers plus retained-size aggregated by constructor. Self_size ranking is blind to small containers pinning huge subgraphs; retained size is what named SubAgentWatcher ×7 = 263 MB / 88%. Needs node --max-old-space-size=12288 for large snapshots. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ontent, and nested tool_use.input — close residual leak shaps found in review Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This was referenced Jun 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root cause (dominator-tree heap analysis)
A dominator-tree retained-size analysis of the main-process heap snapshot proved the leak definitively: 7 live
SubAgentWatcherinstances retained 263 MB = 88% of the entire reachable main-process heap, via the chainPR #300 capped the entry COUNT (
MAX_RETAINED_ENTRIES_PER_AGENT=500) but not the per-entry BYTE SIZE. So each of the 500 retained entries per agent still pinned its FULL body — Read tool outputs,tool_resultbodies, the skills-list attachment,[Truncated: PARTIAL view]system-reminders. 500 entries × multi-MB bodies × several agents × 7 watchers = 263 MB, and churning those multi-MB strings on every 600 ms poll fragmented V8 into ~1.5 GB RSS.Part 1 — the fix: truncate-at-retention (zero visible change)
The mini-feed is a liveness affordance, not a transcript archive.
buildSubAgentStateonly reads structure (block types, tool names,tool_use_id,is_error, timestamps) plus an ≤80-char headline per tool call — it never renders a full body. So clamping the large free-text payloads at retention to a cap well above what the renderer shows is invisible to the user while removing the bytes that caused the leak.SubAgentWatcher.ts: newtruncateEntryBodies(entry, 8KB)clamps the large free-text fields (text/thinkingtext,tool_resultcontent/outputin both string and array-of-{text}shapes, largetool_useinputstring values, skillattachment.content, system-reminder text) before pushing intoentriesByAgent, at the single parse/push site. All structural fieldsbuildSubAgentStatereads are preserved. Each clamp appends an honest… [truncated N bytes — full body on disk]marker. 8 KB is ~100× the 80-char headline the renderer derives from any field, so no rendered value can ever be clipped; the durable full transcript stays on disk inagent-<id>.jsonl.codexSubagentState.ts: droppedchildEntriesByAgentIdentirely. PR fix(#288): bound CodexSubAgentTracker retained entries + clear on stop (Codex twin of #300) #317 kept a bounded 500-entry raw-entry TAIL "for a future mini-feed," but nothing on the emit path (or anywhere) read it — write-only dead weight that still pinned every entry's full body in main-process heap. The derivedSubAgentStateinchildStateByAgentIdalready carries everythingemit()ships, so removing the tail is invisible. The fix(#288): bound CodexSubAgentTracker retained entries + clear on stop (Codex twin of #300) #317 full-read derivation logic (correctstartedAt/turnCount/childMeta/tool counts) and the uncapped-count dirty signal are untouched.Part 2 — the tool that proved it:
--ownersdominator modescripts/analyze-heapsnapshot.mjsgains a--ownersmode. The default SUMMARY mode ranks byself_size— useless when the leak is a small container (a Map, a class instance) transitively pinning a huge subgraph.--ownersbuilds the dominator tree (Cooper-Harvey-Kennedy) of the whole reachable graph, computes retained size per node ("how much memory is freed if this object is removed"), and prints (a) top individual retainers by retained size with theirtype:nameand (b) retained size aggregated by constructor/owner name. Needsnode --max-old-space-size=12288for large snapshots.Output on the #288 snapshot, naming the leak no
self_sizeranking could surface:Verification
npx tsc -p tsconfig.node.json→ zero new errors in the touched files (only the pre-existing TS18047 incodexSubagentState.extractCodexChildMeta, untouched by this PR, remains).node --check scripts/analyze-heapsnapshot.mjspasses;--ownersconfirmsSubAgentWatcher ×7 = 263 MB / 88%as the top retainer.Note: the real before/after needs a fresh snapshot taken after a rebuild — run
--ownerson it to confirmSubAgentWatcheris no longer near the top of the retained-by-owner table.🤖 Generated with Claude Code