fix: Context window truncation using `CondensationAction` #7578

csmith49 · 2025-03-28T17:29:33Z

This change is worth documenting at https://docs.all-hands.dev/
Include this change in the Release Notes. If checked, you must provide an end-user friendly description for your change below

End-user friendly description of the problem this fixes or functionality that this introduces.

Give a summary of what the PR does, explaining any non-trivial design decisions.

When an LLM hits a context window limitation, the agent controller chops the history roughly in-half so the agent can continue to make forward progress. This behavior is a necessary fallback but introduces some strange behavior (#7175, for example) when sessions are reloaded, especially when condensers are in play.

#7353 introduced two new primitives that can help to resolve this issue: the CondensationAction and the View. Instead of relying on the agent controller to maintain an event stream and a history and keep them in-sync modulo any truncations, we can instead

Modify the context truncation behavior to emit a CondensationAction event. This event records which parts of the history should be removed.
Rely on the use of View objects downstream instead of the raw State.history. When given a list of events, the View automatically applies any condensation actions to produce a smaller history that can be used by agents.

This approach effectively stores the context truncation in the event stream/history. If we reload a session or some state gets out-of-sync, we can produce the exact same truncated history directly from the event stream.

(As an added benefit, this uses the same system as condensers, so they shouldn't end up in a position where they have to fight for priority on what/how to truncate.)

To support these changes, this PR makes the following edits:

View definition relocated and openhands.memory sub-module structure slightly reorganized. Necessary to avoid circular imports for the next edit.
State objects have a State.view property which constructs a View directly from the state's history.
Agent controller context truncation updated to emit CondensationAction instead of manually chopping up the history.
Tests updated.
References to state.history in agents updated to state.view.

Link of any specific issues this addresses.

…ving last of the truncation tests

csmith49 · 2025-03-28T23:55:50Z

tests/unit/test_truncation.py

-        assert controller.state.truncation_id is not None
-        assert controller.state.truncation_id > controller.state.start_id
-
-    def test_history_restoration_after_truncation(self, mock_event_stream, mock_agent):


This test was moved to test_agent_controller.py with no change.

csmith49 · 2025-03-28T23:56:42Z

tests/unit/test_truncation.py

-
-
-class TestTruncation:
-    def test_apply_conversation_window_basic(self, mock_event_stream, mock_agent):


This test was moved to test_agent_controller.py, and extended with the checks from test_context_window_exceeded_handling.

csmith49 · 2025-03-28T23:57:12Z

tests/unit/test_truncation.py

-        assert len(controller.state.history) == history_len
-        assert len(controller.get_trajectory()) == history_len
-
-    def test_context_window_exceeded_handling(self, mock_event_stream, mock_agent):


This test was folded into test_apply_conversation_window_basic during the move.

csmith49 · 2025-03-28T23:59:00Z

tests/unit/test_truncation.py

-            if isinstance(event, CmdOutputObservation):
-                assert any(e._id == event._cause for e in truncated[: i + 1])
-
-    def test_truncation_does_not_impact_trajectory(self, mock_event_stream, mock_agent):


The check for this test was folded into test_context_window_exceeded_error_handling (that's not the test originally from this file, note the extra error in the name).

tests/unit/test_condenser.py

csmith49 · 2025-03-29T00:06:56Z

To anyone looking at this code: I recommend starting with the View class to see how CondensationAction events are used, then taking a look at the changes in the agent controller.

The other changes are important but mostly serve to test/support the above.

neubig

Overall, this looks good to me!

I have a slight concern about the efficiency implications of calling View.from_events many times, but maybe it's not a major concern.

tests/unit/test_agent_controller.py

csmith49 · 2025-03-31T13:27:43Z

Overall, this looks good to me!

I have a slight concern about the efficiency implications of calling View.from_events many times, but maybe it's not a major concern.

Hmm, I see what you mean. I think we'll always have to call it once per-step, but since it's linear in the size of the history that's probably okay. I can optimize away any more calls by caching the view construction in the state.

Co-authored-by: Graham Neubig <neubig@gmail.com>

enyst

Very nice PR, thank you for this! I now see better what you had in mind. 👍

openhands/controller/state/state.py

openhands/controller/agent_controller.py

openhands/controller/state/state.py

enyst · 2025-03-31T19:18:34Z

I tried to play with older streams, and it looks good. I had some fun with Haiku-generated summaries for 400+ events - amazingly it did mostly do a summary indeed, although it didn't respect the format. Sometimes when it's large enough, Sonnet gets taken away by the events and forgets to summarize. 😅

It did indeed get some truncation_id without blinking.

I didn't try streams generated on branch (just a few older ones), I think it's fine though.

csmith49 · 2025-03-31T19:28:06Z

I tried to play with older streams, and it looks good. I had some fun with Haiku-generated summaries for 400+ events - amazingly it did mostly do a summary indeed, although it didn't respect the format. Sometimes when it's large enough, Sonnet gets taken away by the events and forgets to summarize. 😅

There's a few instances we've seen where the summarization forgets pretty important bits. If you're good with these changes LMK -- once I get this merged I'm going to start improving the summarization prompt.

enyst

Just to note, it looks like it also works as expected with no condenser enabled/found.

Thank you for this, I think it's absolutely worth taking this in asap and have fun. ☕️

li-boxuan · 2025-04-01T01:03:12Z

Sorry I didn't get a chance to review carefully but I skimmed over it and didn't spot any major issue. Thanks for the fix!

…7578) Co-authored-by: Calvin Smith <calvin@all-hands.dev> Co-authored-by: Graham Neubig <neubig@gmail.com>

Calvin Smith and others added 12 commits March 27, 2025 19:46

view property on state (resolves circular import)

1957417

truncation via condensation

ce0f242

test framework for truncation, test not passing (but also not testing)

cadea2e

testing that controller handles condensation from context window

8379992

moving _apply_conversation_window test:

dec534e

moving context_window_exceeded_handling into existing test

7dd8e87

moving history restoration test

659c354

moving post-controller close checks to agent controller test and remo…

8271c32

…ving last of the truncation tests

cleaning up condenser interface to be view-centric

3df14e7

fixing condenser unit tests to use new view interface

574bcc0

moving agents -> state utility functions to reference view instead

3e72a36

Merge branch 'main' into fix/truncation-to-condensation

1bba5df

csmith49 marked this pull request as ready for review March 28, 2025 23:52

csmith49 commented Mar 28, 2025

View reviewed changes

tests/unit/test_condenser.py Show resolved Hide resolved

Merge branch 'main' into fix/truncation-to-condensation

e1bbc1a

Merge branch 'main' into fix/truncation-to-condensation

17c3b58

csmith49 requested review from enyst and li-boxuan March 30, 2025 14:28

Merge branch 'main' into fix/truncation-to-condensation

3ec3847

neubig approved these changes Mar 31, 2025

View reviewed changes

tests/unit/test_agent_controller.py Outdated Show resolved Hide resolved

csmith49 and others added 4 commits March 31, 2025 07:29

Update tests/unit/test_agent_controller.py

90b9fee

Co-authored-by: Graham Neubig <neubig@gmail.com>

Merge branch 'main' into fix/truncation-to-condensation

71c6301

Merge branch 'main' into fix/truncation-to-condensation

8e7740b

view caching on state

93f8b5d

enyst reviewed Mar 31, 2025

View reviewed changes

openhands/controller/state/state.py Show resolved Hide resolved

openhands/controller/agent_controller.py Show resolved Hide resolved

Calvin Smith added 2 commits March 31, 2025 10:37

testing state view caching

467edeb

removing unused references to state.truncation_id

3788dd4

enyst reviewed Mar 31, 2025

View reviewed changes

openhands/controller/state/state.py Show resolved Hide resolved

enyst approved these changes Mar 31, 2025

View reviewed changes

csmith49 merged commit abaf0da into OpenHands:main Mar 31, 2025
24 checks passed

doew pushed a commit to doew/OpenHands that referenced this pull request Apr 2, 2025

fix: Context window truncation using CondensationAction (OpenHands#…

9f971c5

…7578) Co-authored-by: Calvin Smith <calvin@all-hands.dev> Co-authored-by: Graham Neubig <neubig@gmail.com>

csmith49 mentioned this pull request May 30, 2025

[Bug]: Endless “CondensationAction” Loop Caused by Constant Context Overflow #8630

Closed

1 task

shabbir-shakudo pushed a commit to devsentient/OpenHands that referenced this pull request Jul 15, 2025

fix: Context window truncation using CondensationAction (OpenHands#…

91b3482

…7578) Co-authored-by: Calvin Smith <calvin@all-hands.dev> Co-authored-by: Graham Neubig <neubig@gmail.com>



		class TestTruncation:
		def test_apply_conversation_window_basic(self, mock_event_stream, mock_agent):

fix: Context window truncation using CondensationAction #7578

fix: Context window truncation using CondensationAction #7578

Uh oh!

Conversation

csmith49 commented Mar 28, 2025

Uh oh!

csmith49 Mar 28, 2025

Choose a reason for hiding this comment

Uh oh!

csmith49 Mar 28, 2025

Choose a reason for hiding this comment

Uh oh!

csmith49 Mar 28, 2025

Choose a reason for hiding this comment

Uh oh!

csmith49 Mar 28, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

csmith49 commented Mar 29, 2025

Uh oh!

neubig left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

csmith49 commented Mar 31, 2025

Uh oh!

enyst left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

enyst commented Mar 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

csmith49 commented Mar 31, 2025

Uh oh!

enyst left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

li-boxuan commented Apr 1, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

fix: Context window truncation using `CondensationAction` #7578

fix: Context window truncation using `CondensationAction` #7578

enyst commented Mar 31, 2025 •

edited

Loading