v0.3.1 — streaming-only + token cap fix
Two operator-facing fixes.
Fixed
- Streaming-only. The legacy
/v2HTTP poll loop is retired (the grid's/v2API is gone and returns410).GRID_STREAMINGnow defaults totrue, every start path uses the multi-backend supervisor, and the dead/v2client is removed. Workers that were silently polling/v2now connect over WebSocket. Update to fix the410 Goneerrors. - No more token cap. The worker was clamping every request's
max_tokensdown toGRID_MAX_LENGTH(4096) — cutting long outputs off mid-sentence and starving multi-step tool calls. It now honors the client's requestedmax_tokensas-is and only falls back to a default when none is given. Defaults raised to 32768 output / 131072 context.
Notes
- Multi-backend: every backend in
GRID_BACKENDSis served (set up via the quickstart, each validated live). - To allow even larger single outputs, set
GRID_MAX_LENGTH. Existing single-backend.envconfigs keep working.