[Bug] Generated catalog contains models where maxTokens exceeds contextWindow (Inkling-Small, openrouter gpt-3.5-turbo-0613) #1680
aniruddhaadak80
started this conversation in
Bug reports
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Area
AI providers and models
Description
Two entries in the generated model catalog (
packages/ai/src/models.generated.ts, currentmain) declaremaxTokenslarger than their owncontextWindow, which is impossible for any single request:thinkingmachines/Inkling-Smallopenai/gpt-3.5-turbo-0613For comparison, the sibling entry
thinkingmachines/Inkling(huggingface) declarescontextWindow: 1048576withmaxTokens: 1048576, soInkling-Small's output limit looks copy-pasted from its larger sibling while its window is half the size. The openrouter entry is off by exactly one token (4096 > 4095), suggesting an upstream rounding/data quirk that the generator copied verbatim.Both values flow into request construction:
packages/ai/src/providers/simple-options.ts:6derives the default output budget frommodel.maxTokens(clamped to 32000, which masks the problem in the default path), andpackages/ai/src/providers/openai-completions.ts:540sendsoptions.maxTokensverbatim when a caller sets it explicitly. So today the practical blast radius is limited to SDK/extension callers passing explicit budgets and to anything consuming these fields as metadata - but the invariant itself is broken, and nothing currently catches it when upstream data regenerates.Steps to reproduce
Or mechanically: scan all
} satisfies Model<...>entries and assertmaxTokens <= contextWindow.Expected behavior
Every catalog entry satisfies
0 < maxTokens <= contextWindow. Values that cannot be satisfied from upstream data should be clamped or overridden inpackages/ai/scripts/generate-models.tswith a comment explaining why.Actual behavior
maxTokens > contextWindowon the two entries listed above; requests built from explicitmaxTokensin that range are guaranteed to be rejected by the provider.Prime Agent version
main @ e319a66
Environment
Any (generated data; platform-independent)
Additional context
I first filed this as issue #1677; it was auto-closed by the contribution vouch gate within seconds, so I am re-filing it here per CONTRIBUTING.md's discussion-first process.
Suggested fix: add an assertion (or normalize-and-warn step) in the generator so regeneration fails or corrects instead of silently emitting impossible limits, then override/clamp the two known offenders.
All reactions