Condensing depends on a context ceiling it can't see (#11801, #7992, #6396 look like one bug) #14617
behrnt-slatgng
started this conversation in
1. Feature requests
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Disclosure up front: I build Grunz, a hosted chat + coding agent on open-weight models, so I'm working on the same problem as you. I'm not selling anything here. Condensing has been the hardest part of our agent to get right, and three of your closed issues describe the same failure from different angles, so here is how they seem to connect.
Condensing depends on a ceiling it can't see
The first two are the same bug. The trigger compares usage against a declared window, and the declared window is a property of the weights. The window you are actually served belongs to whoever serves them. On the hosted endpoints I have checked, most serve about 32K no matter what the model card says (the card figures run 128K to 1M), and few serve more than 256K. With a ceiling that's too high, condensing never triggers. With one that's too low, it triggers too early. And when the provider rejects the request, the recovery path runs at the worst possible moment, when the prompt is already over the limit.
The 207K to 589K jump looks like the condense request itself was built from the full history plus the condensing prompt, so it could never fit under the ceiling it was trying to get back under.
Condensing is a lossy write, and the goal goes first
#6396 is the other half. A summarizer keeps whatever looks salient in the transcript, and the original goal is usually the oldest and least-repeated text in it. So it gets squashed into a topic, the agent sees a stated task, and it starts over. We hit the same thing, and our first fix made it worse: we pinned the goal by adding it to the system prompt, which counts toward the same budget. That moved the trigger earlier, and on a 32K endpoint the run started condensing in a loop. What did work was keeping the goal as a small, separately budgeted object that the summarizer can't rewrite, and never feeding it back in as something to summarize.
Questions
All reactions