Skip to content

fix(core): bound per-agent image memory (proactive budget + inherited-context scrub)#779

Merged
0xallam merged 6 commits into
mainfrom
devin/1784159809-bound-agent-image-memory
Jul 16, 2026
Merged

fix(core): bound per-agent image memory (proactive budget + inherited-context scrub)#779
0xallam merged 6 commits into
mainfrom
devin/1784159809-bound-agent-image-memory

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Summary

Screenshot/image tool outputs accumulated without bound in agent context, so long deep scans grew process RSS roughly linearly (a handful of very large base64 blobs, flat Python object count) until the host OOM-killed the run. Two independent leaks, two fixes.

1. Images were never pruned from a running agent's session.
view_image returns up to a 10 MB base64 input_image block. It persists in the agent's SQLiteSession for the whole run and is re-materialised into RAM and re-sent to the model on every turn. The only existing defense (strip_all_images_from_session) is reactive — it fires only after the provider rejects a request (400/404/422) and is all-or-nothing.

New enforce_image_budget(session, max_images) keeps the most recent max_images image outputs and replaces only the older input_image blocks with a text placeholder (sibling text in the same tool result is preserved). Called before each turn in the run loop.

Bound is configurable: RuntimeSettings.max_context_images (STRIX_MAX_CONTEXT_IMAGES, default 3, 0 keeps none, negative rejected via ge=0), threaded into the run via context["max_context_images"].

2. create_agent(inherit_context=True) duplicated every screenshot into each child.
On spawn, the parent's entire turn input is json.dumps-ed verbatim into the child's first message. That inlined every base64 screenshot as text — duplicated across all N children — and, once serialized into a text blob, it could no longer be reclaimed by the reactive strip (which only matches structured input_image blocks).

New scrub_images_from_items() replaces image blocks with a text placeholder before serialisation:

# inputs.py, child_initial_input
rendered = json.dumps(scrub_images_from_items(parent_history), ...)

Net effect: per-agent image memory is capped at max_context_images live screenshots, and inherited context carries none. Text/reasoning history is untouched.

Concurrency safety

Session rewrites (enforce_image_budget / strip_all_images_from_session) read a snapshot, then clear_session() + add_items(). AgentCoordinator.send() also appends peer/user messages to a session from another task. Both now acquire one shared per-session session_write_lock, so a rewrite built from a snapshot can never clear away a concurrently-delivered message. On a failed re-add the original items are restored (logged, not silently suppressed) so a rewrite can't leave history empty.

Changes

  • strix/core/sessions.py — add enforce_image_budget, scrub_images_from_items, session_write_lock, and shared _output_has_image / _elided_output / _rewrite_session helpers; refactor strip_all_images_from_session onto them.
  • strix/core/agents.py — acquire session_write_lock around out-of-band send() appends.
  • strix/core/execution.py — enforce the budget before each turn (best-effort; failures logged, never fatal).
  • strix/core/inputs.py — scrub images out of inherited child context.
  • strix/core/runner.py — pass max_context_images into the run context.
  • strix/config/settings.pymax_context_images setting (STRIX_MAX_CONTEXT_IMAGES, default 3, ge=0).

Notes

  • Behavioural: agents keep only the most recent N screenshots in context; older visual detail is dropped (text placeholder retained). Default 3 is tunable.
  • ruff, ruff-format, mypy, bandit, and the touched test suites pass locally via pre-commit.

Link to Devin session: https://app.devin.ai/sessions/dad023e379e942f287bcf6822463b7a4
Requested by: @0xallam

@0xallam 0xallam self-assigned this Jul 15, 2026
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@greptile-apps

greptile-apps Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR limits image memory in agent sessions and inherited context. The main changes are:

  • Adds a configurable limit for retained context images.
  • Replaces older image blocks while preserving sibling text.
  • Removes images before serializing inherited child context.
  • Adds locking and recovery around session rewrites.
  • Passes the image limit through root and child run contexts.

Confidence Score: 5/5

No additional blocking issue was found in the latest changes.

  • The image limit is validated before entering the run context.
  • Mixed image and text outputs now retain their text blocks.
  • Inherited context removes image payloads before serialization.

Important Files Changed

Filename Overview
strix/config/settings.py Adds a validated non-negative setting for the number of retained context images.
strix/core/agents.py Serializes coordinator session writes with the shared session lock.
strix/core/execution.py Enforces the configured image budget before each agent turn.
strix/core/inputs.py Removes structured image blocks before inherited context is serialized.
strix/core/runner.py Adds the configured image limit to the agent run context.
strix/core/sessions.py Adds image elision, inherited-context scrubbing, shared locking, and session rewrite recovery.
tests/test_runner_rate_limit.py Updates the mocked runtime settings for the new image limit.

Reviews (3): Last reviewed commit: "docs(core): trim verbose comments/docstr..." | Re-trigger Greptile

Comment thread strix/core/sessions.py
Comment thread strix/core/sessions.py Outdated
Comment thread strix/config/settings.py Outdated
0xallam added 3 commits July 16, 2026 00:04
- preserve sibling text/non-image blocks when eliding an image output
- serialise session rewrites with a per-session lock and restore original
  items on failure so a rewrite can never drop or empty history
- reject negative STRIX_MAX_CONTEXT_IMAGES (ge=0) so it can't silently
  disable the budget
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptile

Comment thread strix/core/sessions.py Outdated
0xallam added 2 commits July 16, 2026 01:01
The image-budget rewrite read a snapshot then cleared+re-added, but
AgentCoordinator.send() appends peer/user messages to the same session
from another task without that lock, so a message delivered mid-rewrite
could be clobbered. Expose session_write_lock() and acquire it in both
the rewrite and send() paths. Also stop silently suppressing restore
failures on a failed re-add (log + re-raise instead).
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptile

@0xallam
0xallam merged commit 899e07d into main Jul 16, 2026
2 checks passed
@0xallam
0xallam deleted the devin/1784159809-bound-agent-image-memory branch July 16, 2026 01:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant