fix(app): drop retained base64 screenshots from side panel chat state

[Dani Akash (DaniAkash)]

Problem

The Assistant side panel grows to multiple GB during active use and eventually OOMs the renderer, auto-closing the panel (#1972). One contributor is that the AI SDK keeps the entire conversation resident in memory for the whole session, and on the default provider path browser tool results carry inline base64 screenshots (structured image data). The side panel never renders those images (the chat only shows a tool label), so the screenshots accumulate in the tab's heap, in the resident set of recent conversations, and in persisted history, with nothing releasing them.

Change

Add a small pure helper (tool-output-strip.ts) that replaces base64 image blocks inside tool-result outputs with a short text marker, preserving everything else (text blocks, isError, and output metadata such as the active tab id). It returns the same array reference when there is nothing to strip so callers can skip a redundant update.

Apply it in two places:

• When building the persistable messages for a conversation, so saved and reloaded history no longer carries screenshot bytes.
• After a turn settles (not while streaming), strip older messages in the live useChat state while leaving the latest turn intact.

Why it is safe

• Nothing in the side panel renders tool-output images, so removing the bytes has no visible effect on the transcript. Non-image tool output, text, and metadata are untouched.
• Multi-turn context is unaffected: the client already sends only a bounded, text-only history plus the new message to the server, which keeps the working conversation state itself. The transcript is not the source of truth for context, so trimming retained outputs does not change what the model sees.
• The in-memory pass runs only when a turn is no longer streaming and keeps the most recent message intact, matching the existing empty-parts cleanup.

Scope / notes

• This mainly helps the default / BYOK provider path, where tool outputs carry base64 inline. On ACP named-agent providers tool outputs are already text-only.
• User-attached images are out of scope here.

Validation

• New unit tests for the helper (image stripping, metadata preservation, keep-last-message, identity when nothing to strip, non-object output). Existing persistence tests still pass.
• apps/app typecheck passes (after GraphQL codegen); Biome clean on the changed files.

[greptile-apps]

Greptile Summary

This PR removes retained base64 screenshots from side-panel chat messages. The main changes are:

• Adds a copy-on-write helper for replacing tool-output images with a text marker.
• Strips screenshots before saving local or remote conversation history.
• Cleans older live messages after streaming finishes.
• Adds unit coverage for stripping, metadata, identity, and unsupported outputs.

Confidence Score: 5/5

This looks safe to merge.

• No blocking issues found in the changed code.
• The helper matches the screenshot shape produced by the browser tools.
• The state cleanup stops after one update because unchanged messages retain their array reference.

Important Files Changed

Filename	Overview
packages/browseros-agent/apps/app/modules/chat/tool-output-strip.ts	Adds defensive, immutable stripping for the screenshot output shape produced by the browser tools.
packages/browseros-agent/apps/app/modules/chat/chat-session.hooks.ts	Cleans older tool outputs after streaming and avoids redundant message-state updates.
packages/browseros-agent/apps/app/modules/chat/chat-session-persistence.ts	Removes screenshot bytes from persisted messages after filtering empty entries.
packages/browseros-agent/apps/app/modules/chat/tool-output-strip.test.ts	Covers image replacement, metadata preservation, latest-message retention, identity, and unsupported outputs.

_{Reviews (1): Last reviewed commit: "fix(app): drop retained base64 screensho..." | Re-trigger Greptile}

[github-actions]

✅ Tests passed — 1635/1639

Suite	Passed	Failed	Skipped
✅ agent	344/344	0	0
✅ build	34/34	0	0
✅ claw-app	185/185	0	0
⚠️ claw-mcp	0/0	0	0
⚠️ claw-onboard	0/0	0	0
⚠️ claw-server-rust-quality	0/0	0	0
⚠️ claw-server-rust	0/0	0	0
✅ server-agent	314/314	0	0
✅ server-api	147/147	0	0
✅ server-browser	10/10	0	0
✅ server-integration	10/10	0	0
✅ server-lib	299/300	0	1
✅ server-root	38/41	0	3
✅ server-tools	254/254	0	0

View workflow run