[Bug] llm-pi-ai silently resolves an unconfigured route to a 262144 context window while the provider declares 1048576

[gorban]

Summary

llm-pi-ai resolves an unconfigured route's context window from DEFAULT_CONTEXT_WINDOW (262,144) even when the provider publishes a larger capacity for the same model id. In this session two routes on the same profile disagreed:

route	harness-resolved contextWindow	provider-declared
modlens-openrouter/deepseek-v4-flash	1,024,000	1,024,000
openrouter/deepseek/deepseek-v4.1-flash	262,144	1,048,576

The profile declares the model with only id and name, so the default applies silently. Compaction then fires ~4× early, and the harness infers context overflow at 262,144 while the provider accepts 1,048,576.

Environment

• dsh 0.1.6-alpha.2, commit ddefc45fbc7f8e46dd73185e68295696d1297887
• ~/.dsh/settings.yaml:

llm-pi-ai:
  providers:
    openrouter:
      apiKeyEnv: OPENROUTER_API_KEY
      api: openai-completions
      baseURL: https://openrouter.ai/api/v1
      models:
        - id: deepseek/deepseek-v4.1-flash
          name: Deepseek Flash v4.1

No defaultContextWindow, no per-model contextWindow, no built-in catalog entry for this id.

Current behavior

The harness-resolved capacity is recorded in the session's request/context events:

jq -c 'select(.type=="request/context") | .data' session-2496881f-sanitized.jsonl

{"provider":"modlens-openrouter","model":"deepseek-v4-flash","contextWindow":1024000}
{"provider":"modlens-openrouter","model":"deepseek/deepseek-v4.1-flash","contextWindow":262144}
{"provider":"openrouter","model":"deepseek/deepseek-v4.1-flash","contextWindow":262144}

The provider's own catalogue for that exact id, from https://openrouter.ai/api/v1/models:

deepseek/deepseek-v4.1-flash | context_length=1048576 | max_completion=384000

Two consequences in this session:

1. resolveCompactSpec derives thresholdTokens = 0.8 × 262144 = 209,715 and retainTokens = 0.16 × 262144 = 41,943, so pressure compaction starts at a quarter of the usable context.
2. isContextOverflow(message, contextWindow) is called with 262,144, so the harness declares CONTEXT_WINDOW_EXCEEDED for a ~287k-token prompt that the provider served without complaint.

Expected behavior

The default should not silently stand in for a capacity the provider publishes:

• resolve contextWindow from provider metadata when the route is reachable (the discoverModels seam already reads context_length), or
• require an explicit contextWindow/defaultContextWindow for a model id with no built-in entry, failing loud at load, or
• at minimum log a warning when the default is applied to a route whose provider declares a different capacity.

A 4× under-declaration is not a conservative fallback: it changes compaction scheduling, and it makes the harness reject requests the provider would accept.

Relation to existing reports

• Compaction threshold is scaled against the full context window, so it can sit above what the provider admits #5123 and compaction-basic: default thresholdRatio 0.8 can never trigger when the per-request output reserve is large (sessions die before first compaction) #6671 cover the threshold ratio versus the per-request output reserve — a different quantity.
• Context window should follow selected model (Flash/Pro/Ollama all show placeholder 14745) #3684 covers the indicator showing a placeholder window; Proposal: session-level context window override in the model menu #1495 asks for a session-level override. Neither addresses the resolved capacity used for compaction and overflow detection.

Evidence bundle

https://gist.github.com/gorban/4725c87390cb391047a2f14b56b98d28 — EVIDENCE.md section 2, plus the full session log.

[gorban]

Refined root cause, from a second session plus a run of the current resolution path against a real settings file — this is narrower than "provider default applied wholesale".

The installed catalog simply has no entry for this id, while three sibling ids are described:

model id	resolved contextWindow	where the number came from
deepseek/deepseek-v4-pro	1,024,000	installed pi-ai catalog
deepseek/deepseek-v4-flash-vision-exp	1,048,576	installed pi-ai catalog
deepseek/deepseek-v4.1-flash	262,144	DEFAULT_CONTEXT_WINDOW fallback
deepseek/deepseek-v4-flash	1,024,000	installed pi-ai catalog

OpenRouter declares context_length: 1048576 for all four. So the failure mode is a catalog miss for a configured model id, which resolves "successfully" to a hard-coded 262,144 — four times smaller, with no diagnostic on the route and nothing in the UI to distinguish it from a declared capacity.

Local workaround (documented field, no code change)

llm-pi-ai:
  providers:
    openrouter:
      models:
        - id: deepseek/deepseek-v4.1-flash
          name: Deepseek Flash v4.1
          contextWindow: 1048576

Verified through resolveProfiles → resolveRouteModels → piProvider.getModels() against the live settings file: the route now materializes contextWindow: 1048576. Hot reload picks it up (settings-file watches with watch: true, and agent.ts re-appends request/context whenever the capacity differs from the last record), so an open session re-records 1048576 on its next request.

Three things that a harness-side fix could cover

1. A catalog miss is silent. The resolved profile carries catalogError and modelErrors for model-level diagnostics, but a configured id that the installed catalog does not describe produces neither. It is indistinguishable from a model whose capacity genuinely is 262,144.
2. The fallback constant is also a floor. It under-declares any model nobody listed — which today means newly released large-context models, exactly the ones most likely to be added by id. The domain could be inverted: a cap that only applies where a provider-agnostic guess is safe, rather than a default applied everywhere.
3. Capacity is not the only field lost. The same fallback gives maxTokens: 32,768 against the catalog's 384,000 for these routes, so a catalog miss under-declares output capacity too. Worth stating explicitly for the workaround: I deliberately did not set maxTokens, because an entry-level maxTokens lands in configuredMaxTokens and starts sending a per-request cap — and with the harness not reserving output against the window, a 384,000 cap would shrink the usable prompt rather than help it.

(Separately, 1,024,000 vs 1,048,576 for the two catalog-described siblings is pi-ai catalog drift; harmless at ~2%.)

The two sessions cited: session-43d62239-cc06-40e0-b72a-8eb3e90ff5fd and session-2496881f-8d36-4ce6-93d9-1954a31930c7.

[argszero]

Your refinement checks out against the installed build. Two things I can add — a structural reason the miss cannot be diagnosed, and a consequence that reaches the wire.

Verification

@earendil-works/pi-ai@0.85.1's openrouter catalog (366 entries) describes:

deepseek/deepseek-v4-flash               ctx 1024000  maxTok 384000
deepseek/deepseek-v4-pro                 ctx 1024000  maxTok 384000
deepseek/deepseek-v4-flash-vision-exp    ctx 1048576  maxTok 384000
~deepseek/deepseek-v4-flash-latest       ctx 1048576  maxTok 131072

and has no deepseek/deepseek-v4.1-flash. The miss is on the newest id of the family, which is the systematic direction you describe: a catalog snapshot can only lag, so the ids most likely to be configured by hand are exactly the ones it cannot size.

Why the miss is structurally undiagnosable

packages/llm/llm-pi-ai/src/catalog.ts:901:

const contextWindow = entry.contextWindow ?? base?.contextWindow ?? request.defaultContextWindow

The invalid() throws inside resolveEntry fire only for a fact with no default — api (catalog.ts:889) and baseURL (:894) — because a route that answers neither cannot be constructed at all. Capacities are schema-defaulted (defaultContextWindow/defaultMaxTokens, config.ts:330-331), so the capacity branch can never yield undefined; "this model declares no capacity" is not a representable state and therefore never reaches catalogError/modelErrors, which carry resolution failures — and a fallback is by construction a success.

A diagnostic needs no new plumbing though: at catalog.ts:901 the condition entry.contextWindow === undefined && base?.contextWindow === undefined is exactly miss-and-default, with both facts in hand, and RouteCatalog already returns a diagnostic map (modelErrors) consumed by callers that would consume a sibling "defaulted capacities" map unchanged.

A fourth consequence, and the most severe: it reaches the wire

The resolved number is not only a scheduling input. It is handed to pi-ai as model.contextWindow (adapter.ts:380 → streamSimple), and pi-ai clamps the request against it — buildBaseOptions → clampMaxTokensToContext (dist/api/simple-options.js:2-9):

const available = model.contextWindow - estimateContextTokens(context).tokens - 4096;
return Math.min(maxTokens, Math.max(1, available));

Once the conversation outgrows the believed window, available is negative and Math.max(1, …) makes dsh ask the provider for max_tokens: 1. In your session that is exactly the one-token length stop recorded in #7214: 287,113 estimated prompt tokens against 262,144 → −29,068 headroom → 1. So the 4× miss does not merely compact early — past the believed window it silently rewrites every request into one nobody can win, and the MIN_MAX_TOKENS = 1 floor (why the output is one token and not zero) is precisely what defeats the overflow detector downstream. That gives the workaround a justification beyond compaction scheduling.

Footnotes on your third point

defaultMaxTokens' docstring says "This sizes the model; it never becomes a per-request cap on its own" (config.ts:134-137). True of dsh, but pi-ai's buildBaseOptions resolves options?.maxTokens ?? model.maxTokens, so the fallback 32,768 does reach the wire on every request.

And the collapse is a band, not a cliff: at 250,000 anchored tokens available is still 8,046, at 258,000 it is 46, and at ≥ 258,048 (= 262,144 − 4,096) it is 1. dsh's own pressure threshold (0.8 × 262,144 = 209,715) sits below that band, so a request that reaches it means compaction did not land — which is #7212's failure in this same session, and the reason the two reports compound rather than sit side by side.