Skip to content

v0.3.0 — multi-backend quickstart + supervisor

Choose a tag to compare

@halfaipg halfaipg released this 23 Jun 16:42

Multi-backend setup wizard, and the worker now serves every configured backend (not just the first).

Added

  • Multi-backend quickstart. The setup wizard configures several backends in one run (an "add another backend?" loop). Each backend is validated live: the model is picked from the backend's real served list, then a real completion is run against (url, key, model) before it goes on the grid. Stale keys and wrong model names are caught at setup instead of failing silently.
  • Streaming is now the default connection mode. Legacy HTTP polling is opt-in (the /v2 queue is retired).

Fixed

  • run_worker.py and the headless streaming path now launch the multi-backend supervisor (run_workers()), which keeps one connection alive per backend and restarts dead ones. Previously only the first backend in GRID_BACKENDS started, so a 2nd/3rd model never came online.

Notes

  • Existing single-backend .env configs keep working unchanged.