Skip to content

v0.10.1 — async, smart routing, observability, plugins + full-project review

Choose a tag to compare

@0xzr 0xzr released this 03 Jun 06:46

The 0.10 line adds four features and a full-project review pass.

New (0.10.0)

  • Asyncfrom freellmpool import AsyncPool: await pool.aask/achat over httpx.AsyncClient, same failover/cooldown/quota/metrics/cache as the sync Pool.
  • Metrics-aware routing + freellmpool benchmark — per-provider EWMA latency + success rate. Default fair routing sinks failing providers; FREELLMPOOL_ROUTING=fast prefers lowest latency.
  • Observabilityon_event= hook (attempt/success/error/cooldown/exhausted) on both pools; FREELLMPOOL_LOG enables logging in the CLI/proxy.
  • Pluginsregister_provider() / register_adapter() + a freellmpool.providers entry-point group.

Hardened (0.10.1)

A full-project review (Codex full pass + an independent manual pass + a Codex re-verify, reconciled to consensus) fixed:

  • proxy: socket read timeout + daemon worker threads; streaming+tools now carries tool_calls (with per-call index); mid-stream errors close the SSE cleanly; content:null no longer becomes "None"; embeddings honor a pinned provider/model; Anthropic /v1/messages returns Anthropic-shaped errors.
  • quota: cross-process safe (POSIX flock + reload-merge).
  • MCP: JSON-RPC parse/invalid-request/batch per spec.
  • cache: sqlite connections explicitly closed (were leaking fds).

Also renamed the base exception BuffetErrorFreeLLMPoolError (deprecated alias kept).

153 tests, ruff clean. Full notes in CHANGELOG.md.