LiteLLM context-overflow errors not recognized: session hard-fails at context limit instead of compacting

[giova23]

LiteLLM context-overflow errors are not recognized: session hard-fails at the context limit instead of compacting

Setup

prime-agent 0.9.3 (installed via npm), talking to a vLLM server fronted by a LiteLLM proxy (OpenAI-compatible, openai-completions API), model with a 262144-token context window configured via models.json.

Symptom

When the context fills, the session dies with the raw error and never recovers:

Error: 400 litellm.BadRequestError: OpenAIException - Requested token count exceeds the model's maximum context length of 262144 tokens. You requested a total of 270128 tokens: 261936 tokens from the input messages and 8192 tokens for the completion. Please reduce the number of tokens in the input messages or the completion to fit within the limit.. Received Model Group=qwen3.8-27b-nvfp4

Root cause

isContextOverflow() in packages/ai/src/utils/overflow.ts matches OVERFLOW_PATTERNS against the error message. LiteLLM's wording ("Requested token count exceeds the model's maximum context length of X tokens...") matches none of the patterns, so:

1. _isRetryableError() classifies the 400 as retryable and the same over-limit request is re-sent.
2. Overflow recovery in AgentSession._checkCompaction (Case 1) never runs, so no auto-compaction + retry happens.

This is easy to hit in practice: threshold compaction only runs at turn boundaries, so a single turn whose tool outputs grow the context past the hard limit (context window minus max output tokens) produces exactly this unrecoverable error.

Related earlier report of the same failure class: #1490 (a 400 classified retryable and never repaired).

Proposed fix

Add /requested token count exceeds/i to OVERFLOW_PATTERNS, plus a regression test in packages/ai/test/overflow.test.ts with the real error text. The patterns are only evaluated for stopReason: "error" messages, so there is no false-positive risk on success paths.

I had opened PR #2086 with exactly this before finding the vouched-contributor policy; it was auto-closed. The branch with the change (3 files: overflow pattern + comment, test, changelog fragment) is here if a maintainer wants to pick it up:

https://github.com/giova23/prime-agent/tree/fix-litellm-overflow-detection

Happy to iterate here first if you'd like a different approach (e.g. a broader pattern, or handling this in the retry classifier instead).

[kevinjosethomas]

I’m taking a look at this.

[kevinjosethomas]

Merged in #2129; this fix is not released yet.