Skip to content

v0.9.71 — LLM Gateway: BYOK, managed Kortix models, and live cost observability

Choose a tag to compare

@github-actions github-actions released this 21 Jun 20:50
· 4 commits to main since this release
96c2252

LLM Gateway → production

Promotes the full LLM gateway to production — a per-project router with native cost & usage observability, served at gateway.kortix.com.

Highlights

  • Router — resolve → route → bill across OpenAI-compatible, Anthropic, Bedrock and Codex transports, with fallback chains and a circuit breaker.
  • Managed models — Kortix Power (Claude Sonnet) and Kortix Basic (Claude Haiku) on AWS Bedrock.
  • Per-project BYOK — bring your own provider keys, injected per session.
  • Live pricing — every LLM call priced from the live models.dev feed (24h refresh) with a 20% margin; no hardcoded price tables.
  • Observability — Overview, Cost (per-session LLM + sandbox compute), Usage (requests / tokens / latency / errors), Logs, Budgets, API keys, and a Playground.
  • Per-session cost — LLM + sandbox compute attributed per session.

What's Changed

  • chore(release): VERSION → 0.9.71 [skip ci] by @github-actions[bot] in #3638
  • feat(gateway): price LLM usage from live models.dev + 15% margin (no hardcoded pricing) by @lillyboga in #3639
  • chore(dev-eks): deploy dev-d08e2f2f [skip ci] by @github-actions[bot] in #3640

Full Changelog: v0.9.70...v0.9.71