HotKV Gateway 1.1.1

Fixes found by running the gateway against the real OpenAI and Anthropic APIs.

• Anthropic automatic prompt caching now marks what repeats. --anthropic-auto-cache puts breakpoints on the end of the tool definitions and of the system prompt, and adds Anthropic's automatic breakpoint only once a conversation has an earlier assistant turn. In 1.1.0 it marked the last message, so a long fixed system prompt followed by different questions wrote the provider cache on every request and never read it back. A single question with no system prompt and no tools is left alone.
• Semantic threshold default is now 0.90 (was 0.92). Measured with text-embedding-3-small, rephrasings of one question scored 0.81 to 0.94 and unrelated questions below 0.1.
• Semantic reuse only for answers that finished (finish_reason of stop), never for answers cut by the token limit.
• scripts/live_providers.py checks the gateway against the real APIs with your own keys (it costs a few cents).

Checked against the real APIs: OpenAI chat, streaming, embeddings and semantic hits, Anthropic messages and streaming, replay accepted by both official SDKs, and the providers' own prompt-cache usage (Anthropic read back all 13,144 cached tokens on the second request; the gateway's counters matched the API's).

Binaries were built from this tag; Linux amd64 was tested, the other platforms are cross-compiled and untested. Check them with SHA256SUMS.

go install github.com/hotkv-ltd/hotkv-gateway/cmd/hotkv-gateway@v1.1.1

HotKV Gateway 1.1.0

Adds Anthropic's Messages API and makes the providers' own prompt caches visible.

• Anthropic Messages API. Requests to /v1/messages are cached like the OpenAI-compatible ones, streaming included. anthropic-version, anthropic-beta and openai-beta are part of the cache key. count_tokens, batches and other paths pass through untouched.
• Automatic prompt caching (opt-in). --anthropic-auto-cache turns on Anthropic's own prompt caching for requests that do not use it; --anthropic-cache-ttl 1h selects the longer lifetime.
• Provider cache metrics. /_hotkv/metrics now reports the tokens the providers charged, read from and wrote to their own caches, for OpenAI and Anthropic.

Anthropic support was verified with the official Anthropic SDK against a stand-in that follows the documented wire format; it has not yet been run against the live API.

Binaries were built from this tag; Linux amd64 was tested, the other platforms are cross-compiled and untested. Check them with SHA256SUMS.

go install github.com/hotkv-ltd/hotkv-gateway/cmd/hotkv-gateway@v1.1.0

HotKV Gateway 1.0.0

First release of the HotKV Gateway: a drop-in caching proxy for OpenAI-compatible APIs, backed by HotKV.

• Exact cache for chat, completion and embeddings requests, with streaming replay.
• Optional semantic cache (needs a HotKV license that includes it).
• One cache per API key by default; errors and interrupted streams are never cached; a failing cache never fails a request.
• Prometheus metrics at /_hotkv/metrics.

Binaries below were built from this tag. Linux amd64 was tested; the other platforms are cross-compiled and untested. Check them with SHA256SUMS.

go install github.com/hotkv-ltd/hotkv-gateway/cmd/hotkv-gateway@v1.0.0