v0.2.23

LLM Proxy v0.2.23

Compatibility and operational hardening for the maintained pi0n00r fork.

Highlights

• Preserves Ollama request semantics over OpenAI-compatible backends, including JSON/schema output, request-scoped think: false, tool-call IDs and object arguments, and server-managed keep_alive/num_ctx validation.
• Accepts equivalent indefinite keepalive forms such as numeric -1, string "-1", and duration string "-1s", while still rejecting genuinely conflicting residence hints.
• Bounds completion request bodies and returns HTTP 413 before backend work for oversized requests.
• Avoids printing malformed request bodies unless raw request logging is explicitly enabled.
• Validates operational numeric configuration, honors explicit zero cleanup settings, and drains active requests during bounded graceful shutdown.
• Ships a non-root Linux x86-64 container with a packaged health check and Apache-2.0 metadata.

Install

docker pull ghcr.io/pi0n00r/llm_proxy:v0.2.23

The Linux x86-64 archive and its checksum are attached to this release.

Verification

Formatting, vet, the full test suite, and the race detector passed. The published image was independently pulled and validated for non-root, read-only execution; health and model discovery; streamed chat; structured generation; compatibility hints; streamed tool calls and follow-up; metadata-only logging; and oversized-request rejection.

Known Limitations

• LLM Proxy does not provide authentication or TLS. Bind it to loopback or a trusted private network, or place it behind an authenticated TLS reverse proxy. Do not expose it directly to the public internet.
• /health is a liveness check; it does not prove backend readiness or model acceleration.
• Request bodies are bounded, but backend response size is not independently capped. Use a trusted, resource-bounded backend.
• Once a streamed response has begun, a later backend failure can only terminate the stream; it cannot be converted into a conventional JSON error response with a new HTTP status.
• Pre-built artifacts currently target Linux x86-64.

Full changelog: v0.2.22...v0.2.23