freellmpool 0.9.2
Performance: connection pooling.
- Requests reuse a keep-alive httpx client instead of opening a fresh TCP+TLS connection each call — big latency win for repeated calls to the same provider (agent loops, the proxy). ~0.15s/call warm.
- Fast-fail connect timeout (10s) so dead providers fail over quickly.
- Quota ordering snapshots once per request instead of per-candidate locked reads (matters for the 300+ model catalog).
pip install -U freellmpool