Skip to content

1.8.1 — max_tokens no longer capped below real model ceilings

Choose a tag to compare

@VickyXAI VickyXAI released this 21 Jul 05:17

validate_max_tokens rejected anything over 100,000 client-side. That number was not a model limit and not the gateway's — a hardcoded sanity check with no comment that quietly became the binding constraint on every caller. Asking zai/glm-5.2 for the 262,144 it advertises raised a ValueError instead of sending a request.

Probed the live gateway 2026-07-21 with the guard bypassed: 19 models advertise a ceiling above 100,000 and all 19 accepted itzai/glm-5.2 at 262,144, and the whole 128,000 class (claude-opus-4.8, claude-sonnet-5, claude-fable-5, gpt-5.6-sol/terra/luna, gpt-5.5, gpt-5.4, gpt-5.3-codex, glm-5/5.1/5-turbo). Zero rejections.

MAX_TOKENS_SANITY_LIMIT is now 1,000,000, documented as a typo guard. The bound stays so a stray 1e9 or a byte count fails locally rather than becoming a payment quote; it just must never be reachable by a real request.

The old message made this worse: max_tokens too large (maximum: 100000) reads like a provider response, and during an investigation it was taken for exactly that and recorded as an upstream model ceiling in a downstream token table. A test now pins that the message names the SDK's own number and disclaims being a model limit.

373 pass.