Skip to content

0.9.0

Pre-release
Pre-release

Choose a tag to compare

View all tags
@leoguillaume leoguillaume released this 29 Sep 17:50
· 58 commits to main since this release
0.9.0
fa06b35
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

OpenGateLLM 0.9.0 finishes the clean-architecture migration: chat completions join the new stack and the leftover legacy packages are deleted. Every user must belong to an organization. The Celery/QoS/RedisTimeSeries path and the chat search tool are removed. Keys are revoked in place rather than deleted, listed with a status filter, and renamed via PATCH. Model-forward usage is recorded in PostgreSQL by default, or in Langfuse when that dependency is configured — then GET /v1/usage reads Langfuse only. API process logs are JSON. Run the Alembic migrations before starting the new API: they backfill a default organization, drop provider QoS columns, rewrite image-text-to-text routers, add token.revoked, and rename the Postgres modeltype enum to routertype. Drop Celery/qos_* keys from config.yml and replace type: image-text-to-text with text-generation.

Breaking changes

  • Remove provider qos_metric / qos_limit, the Celery routing worker, and Redis TimeSeries. Drop dependencies.celery, routing_max_retries, routing_retry_countdown, and routing_max_priority from config.yml. least_busy uses the Redis inflight gauge only; standard Redis is enough — stop running a routing worker (#1131).
  • Require organization_id on POST/PATCH /v1/admin/users; omitting it or sending null is 422. UserResponse and MeResponse organization_id are no longer nullable. The migration creates a default organization and backfills users without one. Point auth_sso_default_organization_id at an existing organization (#1137).
  • Remove the search tool from POST /v1/chat/completions (tools: [{"type": "search", ...}] and search_results). search_opengaterag_url is ignored if still present (#1138).
  • Replace router type image-text-to-text with text-generation (same /v1/chat/completions surface). Switch leftover type: image-text-to-text in config.yml before start; existing rows are rewritten by Alembic (#1140).
  • Stamp chat, embeddings, OCR, rerank, and audio response id (and streamed chat chunk id) with a 32-character hex usage/trace id instead of request-{uuid}. Streamed chat model is the router name (#1138, #1140, #1142).
  • GET /v1/keys and GET /v1/admin/keys return every key by default, including expired and revoked; filter with status=active|expired|revoked. DELETE revokes the row instead of removing it. Duplicate names no longer return 409 (#1076, #1152).
  • Remove monitoring_postgres_enabled. Usage is always recorded: PostgreSQL by default, Langfuse only when that dependency is configured (no dual-write). GET /v1/usage then reads Langfuse, and traces are named /v1/chat/completions (and the other /v1/... paths) instead of chat-completions (#1142, #1150, #1168).
  • Make POST /v1/ocr model required. Chat, embeddings, and OCR OpenAPI no longer declare extra sampling/tool fields; undeclared body keys are still forwarded without gateway defaults (#1173).
  • Emit API logs as JSON. log_format and log_json are removed (leftover keys are ignored). The uvicorn access log is replaced by an api.request line; set log_config for a custom dictConfig. Every response includes X-Request-ID (#1176).

Features

  • Add GET /v1/organizations/me so an authenticated user can load their organization without the admin API (#1149).
  • Add PATCH /v1/keys/{id} (and admin PATCH) to rename a key, expose revoked on key responses, and filter listings by status (#1076, #1152).
  • Charge router RPM/RPD/TPM/TPD only after a successful provider call, not for 429s or failed inference (#1171).

Refactoring

  • Move POST /v1/chat/completions onto the clean-architecture forwarding stack, including streamed and non-streamed paths (#1138, #1140).
  • Delete the leftover api/schemas, api/helpers, api/clients, api/sql, api/endpoints, and api/utils packages. Rename the Postgres enum modeltype to routertype — apply that migration in a window with no old API process still serving (#1159).

Bug fix

  • Lowercase user emails on create and update (#1126).
  • Reject the reserved key name _system_playground_key with 400 on create (#1134).
  • Persist an HTTP status on failed usage records and measure latency on the recorder; drop unused usage.method (#1165).
  • Always terminate chat streams with data: [DONE] after the usage chunk (#1140).
  • Bill the tokens already delivered when a chat stream is disconnected or the provider fails mid-stream; a failure after the first byte is sent as event: error (#1151).

Playground

  • List expired and revoked keys with badges, filter by status, and revoke via DELETE (#1076, #1152).
  • Require an organization on user forms (#1137).
  • Drop QoS metric and limit fields from provider forms (#1131).

Full Changelog: 0.8.0...0.9.0

  • chore(deps): bump svgo in /docs by @dependabot[bot] in #1114
  • chore(deps): bump js-yaml from 4.3.1 to 4.3.2 in /docs by @dependabot[bot] in #1118
  • chore(deps): bump sharp, @astrojs/starlight, astro, astro-contributors and starlight-ion-theme in /docs by @dependabot[bot] in #1120
  • fix(schemas): lowercase user emails and require model type by @leoguillaume in #1126
  • refacto(qos): remove current QoS, celery worker, and timeseries metrics by @leoguillaume in #1131
  • feat(organization): make organization membership mandatory for users by @tibo-pdn in #1137
  • refacto(chat): refacto POST /v1/chat/completions endpoint toward clean architecture by @benjaminpilia in #1138
  • fix(chat): follow-up fixes after chat completions clean architecture by @leoguillaume in #1140
  • feat(langfuse): trace model-forward requests after clean architecture by @leoguillaume in #1142
  • chore(doc): update generated documentation and release versions by @github-actions[bot] in #1144
  • feat(organization): let authenticated users retrieve their organization by @leoguillaume in #1149
  • feat(keys): display expired keys in the keys list by @lboitel in #1076
  • feat(keys): revoke keys, filter by status, and rename via PATCH by @leoguillaume in #1152
  • feat(usage): persist inference usage with PostgresUsageRecorder by @leoguillaume in #1150
  • refacto: remove all legacy code by @benjaminpilia in #1159
  • fix(keys): reject the names the API upserts for itself by @tanguychenier in #1134
  • fix(usage): persist failure status and measure latency on the recorder by @leoguillaume in #1165
  • feat(usage): use Langfuse as alternative usage data source by @leoguillaume in #1168
  • chore: remove markers by @leoguillaume in #1169
  • chore(doc): update generated documentation and release versions by @github-actions[bot] in #1172
  • refactor(rate-limit): charge router limits from the forwarding use case by @bakr-a in #1171
  • chore: emit api logs as json by @benjaminpilia in #1176
  • fix(chat): bill the tokens already delivered when a stream is interrupted by @benjaminpilia in #1151
  • feat(models): remove all unnecessary schema arguments by @leoguillaume in #1173
  • chore(doc): update generated documentation and release versions by @github-actions[bot] in #1181
  • feat(playground): force ProConnect 2FA via oauth2-proxy alpha config by @leoguillaume in #1175

New Contributors

Contributors

tanguychenier, dependabot, and 5 other contributors
Assets 2
Loading