Skip to content

v0.11.0 — capacity management, context-aware failover, audit hardening

Choose a tag to compare

@0xzr 0xzr released this 06 Jun 10:00

Pooled free-tier LLM gateway — this release adds capacity tooling, context-aware failover, a refreshed model catalog, and a full security/correctness hardening pass.

Highlights

Capacity management (from #7 by @arthurlacoste)

  • freellmpool capacity status — which providers are healthy / near quota / missing a key, right now (local-first).
  • freellmpool providers health — live-probe each configured provider.
  • freellmpool keys status|checklist|add — track keys and configure more (with external-catalog discovery).
  • freellmpool catalog sync|status — advisory external provider catalog. Dashboard now shows capacity.

Context-aware failover — free models have small context windows. freellmpool never truncates your input; instead it learns each model's window from provider errors, escalates only to larger-window models, and raises a clear ContextWindowExceeded (a 413 over the proxy) when nothing fits.

Catalog refresh — re-vetted every model live: 22 paid/dead/gated models disabled, 8 recovered re-enabled, NVIDIA nemotron-3-ultra-550b-a55b added. New maintainer tool scripts/vet_catalog.py.

Hardening (full-codebase audit, signed off by Codex + MiMo 2.5 Pro + MiniMax M3) — never crash on a provider's response shape; a typo in the user providers.toml no longer bricks the tool; constant-time proxy auth; secrets written at 0o600; model discovery won't leak the key on redirects; bounded proxy concurrency; learned context limits floored + TTL'd; cache expiry; and more.

Full detail: CHANGELOG. On PyPI: pip install -U freellmpoolhttps://pypi.org/project/freellmpool/0.11.0/

No new dependencies. 232 tests, CI green on Python 3.11–3.13.