Skip to content

v0.2.2

Choose a tag to compare

@dannote dannote released this 30 Sep 00:09
· 6 commits to master since this release

Added

  • The ReqLLM :llm_proxy provider streams in-process. ReqLLM.stream_text/3 and related calls run through LLMProxy routing, guardrails, accounting, telemetry, fallback handling, and concurrency leases without an HTTP request, and cancelling the stream releases its lease.

Fixed

  • ReqLLM messages and tools are encoded as OpenAI wire data before routing, so normalized ReqLLM.Context requests parse like HTTP requests instead of failing on message structs.
  • OpenAI Codex Chat and Responses requests forward stable prompt_cache_key and session/thread identities scoped to the authenticated API key.
  • OpenAI Codex quota errors keep their upstream status when the error is nested in a WebSocket frame, and status-less quota codes with a future resets_at now set the model-token cooldown and retry delay.
  • OpenAI cached input tokens are normalized once before accounting; raw ReqLLM usage maps are no longer counted twice.
  • Provider-usage refresh no longer fails before dispatch because of an incompatible Finch and connect-options combination in the HTTP builder.