Skip to content

0.8.0

Choose a tag to compare

@leoguillaume leoguillaume released this 21 Sep 09:33
dd49d56

OpenGateLLM 0.8.0 changes usage reporting and token rate limits. GET /v1/usage now returns daily UTC buckets and requires start_time / end_time; list items use "object": "usage.bucket". Key and login responses rename user to user_id. TPM and TPD now count prompt plus completion tokens after the response, and a prompt that does not fit the remaining budget is rejected with 429 before the provider call — existing token quotas will be reached sooner. The playground gets a tighter layout, accurate pagination, and charts for the new usage buckets. Run the Alembic migration that renames the provider hosting-zone enum before starting the new API.

Breaking changes

  • Change GET /v1/usage (and the deprecated GET /v1/me/usage) from per-request rows to daily UTC buckets. start_time and end_time are required. List items use "object": "usage.bucket" with start_time, end_time, token sums, requests, cost, and impacts (#1107).
  • Rename user to user_id on key responses (/v1/keys, /v1/admin/keys) and auth login responses (/v1/auth/login, /v1/auth/sso/login). Create body user and the ?user= filter are unchanged (#1099).

Features

  • Count prompt and completion tokens toward TPM/TPD after the provider response, and reject a request whose prompt does not fit the remaining budget with 429 before the call. Existing TPM/TPD values will be reached sooner (#1077, #1088).

Refactoring

  • Move HTTP encoding off the domain ProviderClient port onto HttpProviderRequest / HttpProviderClient (#1087).
  • Stop importing legacy api/schemas enums from the domain; rename the PostgreSQL enum providercarbonfootprintzone to hostingzone (run the Alembic migration before starting the new code) (#1095).
  • Map domain entities to HTTP responses with from_attributes and drop hand-written field mappers (#1099).
  • Make Router.aliases a non-nullable list (#1112).

Bug fix

  • Make concurrent admin bootstrap idempotent so extra gunicorn workers no longer crash on first start against an empty database (#1073).

Chore

  • Document CLI Python deps and ARM64 compose in the quickstart, and recommend Sentry max_request_body_size: never (#1116).
  • Bump the Langfuse Python SDK to 4.15.1 as a prerequisite for a v4 migration (#1117).
  • Record an ADR for in-request QoS admission without extra workers or RabbitMQ (#1123).

Playground

  • Tighten navigation, page headers, and chat chrome (#1094, #1107).
  • Chart daily usage buckets with date range, summary totals, and peak lines (#1107).
  • Show Page X / Y from the API total field and disable Next on the last page (#983).
  • Fix playground API and Swagger URLs (#1096).
  • Declare jinja2 as a runtime dependency (#1122).

Full Changelog: 0.7.0...0.8.0

  • chore(doc): update generated documentation and release versions by @github-actions[bot] in #1082
  • test(models): autospec the port doubles in the provider request forwarding use-case tests by @tibo-pdn in #1083
  • fix(bootstrap): make concurrent admin bootstrap idempotent across workers by @tibo-pdn in #1073
  • feat(playground): improve playground design by @leoguillaume in #1094
  • fix(playground): wrong url reference and swagger url by @leoguillaume in #1096
  • feat(rate-limiting): reject oversized prompt requests before the provider call by @bakr-a in #1088
  • feat(playground): use API total field to display total pages in pagination by @bakr-a in #983
  • feat(rate-limiting): count output tokens in tpm/tpd and update limits in fastapi hooks by @bakr-a in #1077
  • refacto(domain): remove the domain dependency on legacy schema enums by @tibo-pdn in #1095
  • refacto(schemas): map domain entities to responses with from_attributes only by @tibo-pdn in #1099
  • refacto(router): make Router.aliases non-nullable and align the test factory by @tibo-pdn in #1112
  • feat(usage): return daily usage buckets from GET /v1/usage by @leoguillaume in #1107
  • refacto(provider): move HTTP request details off the ProviderClient port by @leoguillaume in #1087
  • doc(quickstart): document CLI Python deps and ARM64 compose by @leoguillaume in #1116
  • chore: bump Langfuse to latest, prerequisite for V4 migration by @cyrillay in #1117
  • doc(adr): request priorization qos without worker and rabbitmq by @leoguillaume in #1123
  • fix(pyproject): add jinja2 by @benjaminpilia in #1122