v0.10.1 — async, smart routing, observability, plugins + full-project review
The 0.10 line adds four features and a full-project review pass.
New (0.10.0)
- Async —
from freellmpool import AsyncPool:await pool.aask/achatover httpx.AsyncClient, same failover/cooldown/quota/metrics/cache as the syncPool. - Metrics-aware routing +
freellmpool benchmark— per-provider EWMA latency + success rate. Defaultfairrouting sinks failing providers;FREELLMPOOL_ROUTING=fastprefers lowest latency. - Observability —
on_event=hook (attempt/success/error/cooldown/exhausted) on both pools;FREELLMPOOL_LOGenables logging in the CLI/proxy. - Plugins —
register_provider()/register_adapter()+ afreellmpool.providersentry-point group.
Hardened (0.10.1)
A full-project review (Codex full pass + an independent manual pass + a Codex re-verify, reconciled to consensus) fixed:
- proxy: socket read timeout + daemon worker threads; streaming+tools now carries
tool_calls(with per-callindex); mid-stream errors close the SSE cleanly;content:nullno longer becomes"None"; embeddings honor a pinnedprovider/model; Anthropic/v1/messagesreturns Anthropic-shaped errors. - quota: cross-process safe (POSIX flock + reload-merge).
- MCP: JSON-RPC parse/invalid-request/batch per spec.
- cache: sqlite connections explicitly closed (were leaking fds).
Also renamed the base exception BuffetError → FreeLLMPoolError (deprecated alias kept).
153 tests, ruff clean. Full notes in CHANGELOG.md.