Skip to content

v1.0.37 — Fix round batch G: OVERFLOW_TOKEN_SLACK removed, proportional embed shrink, migrate_env hidden-credential note

Choose a tag to compare

@KanenasInGreece KanenasInGreece released this 25 Oct 22:00
· 8 commits to main since this release
ae4efc1

Fix round, batch G: the embed overflow band and the update's credential report. API_VERSION stays 4. Restart the gateway. Update the skills for the version.

Removed

  • OVERFLOW_TOKEN_SLACK is gone (operator ruling decision:3258, measured in fact:3250). This knob set a fixed 16-token band on the save/search path: when the encoder rejected an input as too long for a window that was big enough, the gateway retried only an overshoot inside the band. NREM's get_embedding already shortened and retried any overshoot that was not a window mismatch. The band was never used: no real overflow event appears in the workstation's journal since 2026-09-01, or in G9400's and glxvm's since 2026-10-01. The variable is no longer read, so an install that set it is unaffected, and the entry is removed from .env.example, framework_defaults.py and check_config.py.

Changed

  • When the encoder answers 400 with a parsed "requested N tokens, window W" and the window is big enough, the gateway (_embed, _embed_many) now shrinks and retries every overshoot (N > W). The input shrinks in proportion: len × W ÷ N, never above the reserved clamp, and at least one character shorter than before. If that proportional length would be 0, _embed raises its overrun error at once and posts no shorter string. Before, an overshoot of more than 16 tokens failed the attempt with an "exceeds slack" error. An overshoot within 16 tokens was cut to the smaller of the reserved clamp and one character less than its length, then one character per later retry. The retries stay bounded by EMBED_RETRIES. In a batch, the longest item and every item at or above the reserved clamp shrink; the other items stay whole. If a shrinking item's proportional length would be 0, the batch overrun error is raised and the shrunk batch is not posted. The final error now also names EMBED_CHARS_PER_TOKEN.
  • A parsed 400 that is not an overshoot (N ≤ W) on a big-enough window used to be classified as an overrun and cut to the smaller of the reserved clamp and one character less than its length. It is now an ordinary HTTP error and takes the normal error path. A window smaller than the framework's is still a mismatch and raises the mismatch error, as before.
  • Monitor: encoders.embed_window_overruns_total counts the overrun classifications in the gateway's own _embed and _embed_many. It now counts every overshoot they shrink and retry, including one larger than 16 tokens, which used to fail uncounted. A parsed non-overshoot on a big-enough window is no longer counted; a mismatch was never counted. Key, type and family are unchanged; the telemetry contract's note now reads "every overshoot the gateway snapped and retried". NREM's get_embedding (in the consolidation daemon) now also classifies a large overshoot such as 9000 requested on an 8192 window as overrun. Its own counter is local to that process and does not reach this key.

Fixed

  • migrate_env.py (run by update_framework.sh) no longer shows the gateway's "no usable backend" fallback reason when the unit supplies a provider token through LoadCredential= or a secret Environment= value (register fact:3251). The check runs the gateway's backend loader in a subprocess that cannot see such a credential, so it saw the provider backend as uncredentialed although the running gateway has its token. It now prints one plain sentence instead: such a credential is not visible to this check, so it sees the provider backend as uncredentialed; the running gateway is unaffected and nothing was changed. It prints no credential id, path or value. Its writes still decline on that image, exactly as before. The loader subprocess drops only the gateway's "LLM_BACKENDS_JSON produced no usable backend" log line; every other import-time warning stays in the captured output. The stale comment saying LoadCredential= is invisible to systemctl show is corrected.