Skip to content

v1.3.0

@Mullassery Mullassery tagged this 29 Aug 19:27
vLLM was a cost-estimator-only stub; adds a real backend hitting vLLM's
OpenAI-compatible HTTP API (backends/vllm_backend.rs, engines/vllm_client.rs),
mirroring the existing Ollama backend. Circuit breaker was soft/delayed:
adds a timeout around each provider call, makes the retry loop re-check
health before retrying the same provider instead of after, and adds
half-open auto-recovery after a cooldown.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Assets 2
Loading