freellmpool 0.11.4

MCP Registry verification metadata release.

Added

• Hidden mcp-name: io.github.0xzr/freellmpool README metadata so the official MCP Registry can verify PyPI package ownership.
• Test coverage that freellmpool tokenmax --timeout is also honored by the synthesis request.

v0.11.3 — launch polish and CLI timeout controls

Launch polish and packaging metadata release.

Added

• A CI-backed 30-second quickstart check that installs from a clean venv and
  requires a keyless first reply.
• FAQ.md with prompt-destination, privacy, ToS, failover, ban-risk, and
  comparison answers.
• Above-the-fold README demo assets for tokenmax, plus an upload-ready GitHub
  social preview asset and local GitHub discovery checklist.
• Contributor templates, PR template, and ready-to-file good-first-issue drafts
  for newcomer-sized tasks.
• Release checklist for the operator-run tag/build/publish flow.
• freellmpool ask --timeout and freellmpool tokenmax --timeout so slow
  upstreams can be bounded explicitly from the CLI.

Changed

• Public provider/model counts are now checked from the provider catalog by
  scripts/check-counts, reducing drift across README/docs/release metadata.
• README comparison copy is more explicit and gracious about OpenRouter,
  LiteLLM, and FreeLLMAPI.
• PyPI metadata now has a shorter launch description, broader keywords,
  project URLs, and more complete classifiers.

v0.11.2 — proxy burst backlog fix

Fixed

• Align the local proxy listen backlog with its 128-request worker cap so short mixed-traffic bursts queue cleanly instead of occasionally resetting connections before accept.
• Keep the existing semaphore cap in place so sustained floods remain bounded by the same worker limit.

Release metadata

• Centralize the package version in _version.py and derive the upstream User-Agent from it.
• Align PyPI/docs/README/demo/MCP registry metadata for 0.11.2 and add release metadata drift tests.

Validation

• M3 review: 0 findings.
• Claude Fable review: no blockers.
• CI: Python 3.11, 3.12, and 3.13 passed on PR #42.
• Local: ruff, CI mypy target, 393 tests, build, twine check, and fresh wheel smoke passed.

v0.11.1 — hardening and operations

Patch release after the 0.11 capacity tooling.

Highlights

• freellmpool doctor for no-network local diagnostics.
• Routing-mode normalization and indexed target selection to reduce hot-path lock churn.
• Response cache keys now include routing mode; cache storage uses WAL plus TTL/size pruning.
• Quota counters can batch writes with FREELLMPOOL_QUOTA_FLUSH_EVERY=N.
• Sync and async transports have bounded Retry-After retries, avoid replaying read-phase POST failures, and keep the last provider response when retry delay consumes the deadline.
• Async responses now stream with the same size/deadline guards as sync.
• CI adds catalog validation, focused mypy, coverage ratchet, wheel smoke, and benchmark tooling.
• Proxy Server header now derives from the package version.

Verification

• Claude Fable reviewed the release-prep diff: no blocking findings.
• CI green on Python 3.11, 3.12, and 3.13.
• Local pytest: 390 passed.
• Built sdist/wheel from merged main and smoke-tested the wheel with import, freellmpool --version, and freellmpool doctor.

Full detail: https://github.com/0xzr/freellmpool/blob/main/CHANGELOG.md

v0.11.0 — capacity management, context-aware failover, audit hardening

Pooled free-tier LLM gateway — this release adds capacity tooling, context-aware failover, a refreshed model catalog, and a full security/correctness hardening pass.

Highlights

Capacity management (from #7 by @arthurlacoste)

• freellmpool capacity status — which providers are healthy / near quota / missing a key, right now (local-first).
• freellmpool providers health — live-probe each configured provider.
• freellmpool keys status|checklist|add — track keys and configure more (with external-catalog discovery).
• freellmpool catalog sync|status — advisory external provider catalog. Dashboard now shows capacity.

Context-aware failover — free models have small context windows. freellmpool never truncates your input; instead it learns each model's window from provider errors, escalates only to larger-window models, and raises a clear ContextWindowExceeded (a 413 over the proxy) when nothing fits.

Catalog refresh — re-vetted every model live: 22 paid/dead/gated models disabled, 8 recovered re-enabled, NVIDIA nemotron-3-ultra-550b-a55b added. New maintainer tool scripts/vet_catalog.py.

Hardening (full-codebase audit, signed off by Codex + MiMo 2.5 Pro + MiniMax M3) — never crash on a provider's response shape; a typo in the user providers.toml no longer bricks the tool; constant-time proxy auth; secrets written at 0o600; model discovery won't leak the key on redirects; bounded proxy concurrency; learned context limits floored + TTL'd; cache expiry; and more.

Full detail: CHANGELOG. On PyPI: pip install -U freellmpool → https://pypi.org/project/freellmpool/0.11.0/

No new dependencies. 232 tests, CI green on Python 3.11–3.13.

v0.10.1 — async, smart routing, observability, plugins + full-project review

The 0.10 line adds four features and a full-project review pass.

New (0.10.0)

• Async — from freellmpool import AsyncPool: await pool.aask/achat over httpx.AsyncClient, same failover/cooldown/quota/metrics/cache as the sync Pool.
• Metrics-aware routing + freellmpool benchmark — per-provider EWMA latency + success rate. Default fair routing sinks failing providers; FREELLMPOOL_ROUTING=fast prefers lowest latency.
• Observability — on_event= hook (attempt/success/error/cooldown/exhausted) on both pools; FREELLMPOOL_LOG enables logging in the CLI/proxy.
• Plugins — register_provider() / register_adapter() + a freellmpool.providers entry-point group.

Hardened (0.10.1)

A full-project review (Codex full pass + an independent manual pass + a Codex re-verify, reconciled to consensus) fixed:

• proxy: socket read timeout + daemon worker threads; streaming+tools now carries tool_calls (with per-call index); mid-stream errors close the SSE cleanly; content:null no longer becomes "None"; embeddings honor a pinned provider/model; Anthropic /v1/messages returns Anthropic-shaped errors.
• quota: cross-process safe (POSIX flock + reload-merge).
• MCP: JSON-RPC parse/invalid-request/batch per spec.
• cache: sqlite connections explicitly closed (were leaking fds).

Also renamed the base exception BuffetError → FreeLLMPoolError (deprecated alias kept).

153 tests, ruff clean. Full notes in CHANGELOG.md.

freellmpool 0.9.3

Thread-safety fix from a Codex review of the 0.9.2 connection-pooling change: the pooled httpx client's lazy init is now double-checked-locked (concurrent first requests under the threaded proxy could orphan a client), with atexit cleanup. pip install -U freellmpool

freellmpool 0.9.2

Performance: connection pooling.

• Requests reuse a keep-alive httpx client instead of opening a fresh TCP+TLS connection each call — big latency win for repeated calls to the same provider (agent loops, the proxy). ~0.15s/call warm.
• Fast-fail connect timeout (10s) so dead providers fail over quickly.
• Quota ordering snapshots once per request instead of per-candidate locked reads (matters for the 300+ model catalog).

pip install -U freellmpool

freellmpool 0.9.1

Every model is now live-validated, and the ones that don't work are off by default — so auto-routing only picks models that actually respond.

• All 324 catalog models reachability-tested; 94 broken ones (404/403/402, persistent errors, timeouts) set enabled = false (skipped by auto, still reachable when pinned with -m).
• freellmpool providers shows N (+M off); freellmpool models --all lists disabled ones.
• Fix: reasoning-model token floor 8192→4096 (8192 exceeded Groq's gpt-oss cap); Groq gpt-oss models work again.

pip install -U freellmpool

freellmpool 0.9.0

300+ models + a quota tool over MCP.

• Catalog expanded from 56 to 300+ chat models, discovered live from each provider's /models endpoint and filtered to chat models (embeddings, rerank, audio, and safety classifiers excluded). Not every advertised model is callable on every free tier — freellmpool fails over.
• free_llm_quota MCP tool — today's per-provider usage, daily-limit headroom, session totals, and estimated cost avoided.

pip install -U freellmpool