Skip to content

v0.3.1 — streaming-only + token cap fix

Choose a tag to compare

@halfaipg halfaipg released this 23 Jun 17:04

Two operator-facing fixes.

Fixed

  • Streaming-only. The legacy /v2 HTTP poll loop is retired (the grid's /v2 API is gone and returns 410). GRID_STREAMING now defaults to true, every start path uses the multi-backend supervisor, and the dead /v2 client is removed. Workers that were silently polling /v2 now connect over WebSocket. Update to fix the 410 Gone errors.
  • No more token cap. The worker was clamping every request's max_tokens down to GRID_MAX_LENGTH (4096) — cutting long outputs off mid-sentence and starving multi-step tool calls. It now honors the client's requested max_tokens as-is and only falls back to a default when none is given. Defaults raised to 32768 output / 131072 context.

Notes

  • Multi-backend: every backend in GRID_BACKENDS is served (set up via the quickstart, each validated live).
  • To allow even larger single outputs, set GRID_MAX_LENGTH. Existing single-backend .env configs keep working.