Repository navigation
v1.112.0-dev.2
Pre-release
Pre-release
·
29 commits
to main
since this release
Verify Docker Image Signature
cosign verify \
--certificate-identity https://github.com/BerriAI/project-releaser/.github/workflows/release.yml@refs/heads/main \
--certificate-oidc-issuer https://token.actions.githubusercontent.com \
ghcr.io/berriai/litellm:v1.112.0-dev.2What's Changed
- feat(proxy): let operators omit x-litellm-* response headers by @kenneth-berri in #52949
- feat(apps): link overall app budgets to shared presets by @tin-berri in #52946
- fix(responses): preserve deferred tool references on Anthropic routes by @joshua-berri in #52977
- fix(cost): bill priority rates for image input and cached image tokens by @kenneth-berri in #52972
- fix(bedrock): send the BM25 tool search tool on Claude chat requests by @ryan-crabbe-berri in #52959
- chore(cost-map): add together_ai/tencent/Hy4 from the Together models API by @devin-ai-integration[bot] in #52985
- fix(wandb): set NVIDIA Nemotron 3.5 Super VL output price to $0.47 per 1M by @devin-ai-integration[bot] in #52983
- feat(proxy): emit budget alerts for app customer scopes by @tin-berri in #52955
- feat(proxy): add team budget percentage email alerts by @mrinal-berri in #52859
- fix(caching): preserve otel cache hits and align logging metadata by @joshua-berri in #52971
- feat(ui): configure team budget percentage email alerts by @mrinal-berri in #52860
- fix(bedrock): send native chat and Responses calls to aws_bedrock_runtime_endpoint like Converse by @mika-lt in #50582
- fix(realtime): keep push-to-talk for transcription sessions opened without intent under a transcript guardrail by @quietjules in #50594
- feat(ui): 接入官方简体中文词库 by @le-shi in #48257
- feat(ui): complete zh-CN translations and align terms with the glossary by @yuneng-berri in #52990
- fix(ui): show saved role on user Details without reload by @yuneng-berri in #52991
- feat(proxy): link app member budgets to shared templates by @tin-berri in #52989
- feat(bedrock): derive STS session tags and session name from caller identity by @mateo-berri in #52978
- feat(docker): add a FIPS 140-3 proxy image by @yucheng-berri in #50330
- fix(guardrails): keep tool_result is_error and system row output_config through /v1/messages guardrail rewrites by @ryan-crabbe-berri in #52828
- fix(ocr): read Azure Document Intelligence query options from extra_body by @mika-lt in #51102
- refactor(ui): shrink zh-CN related test runs and write tests in English by @yuneng-berri in #52993
- fix(proxy): decode double-encoded audit log values before masking them by @Aditya-Pathanjali in #50929
- fix(proxy): point the startup schema drift check at the packaged schema.prisma by @narmstrong-berri in #52435
- test(vector-stores): isolate native fallback cooldown state by @devin-ai-integration[bot] in #53002
- test(vertex): align scripted authentication with Rust routing by @devin-ai-integration[bot] in #53001
- test(responses): centralize exact system prompt placement expectations by @devin-ai-integration[bot] in #53000
- feat(redis): serve the dynamic rate limiter's active teams from a native store by @yujonglee-berri in #52995
- chore(model_prices): sync databricks gemini-4-carbon and gemini-4-hydrogen prices by @devin-ai-integration[bot] in #52986
- fix(files): match gemini file id spellings to dispatch parsing by @kerry-berri in #53004
- fix(proxy): compare budget window reset_at as a UTC instant for every duration by @tin-berri in #53012
- fix(proxy): preserve multi-day budget reset offsets by @tin-berri in #53020
- test(e2e): allow a 3-token floor and 10% on provider token counts by @yuneng-berri in #53023
- feat(usage-export): include key and user metadata objects in daily activity export rows by @jesus-berri in #49360
- fix(proxy): drop Redis-only fields from drained window spend by @tin-berri in #53022
- fix(host-http): stop reverse proxies from buffering Rust SSE streams by @yujonglee-berri in #53025
- fix(azure_ai): use the v1 chat path for Azure OpenAI resource bases by @AaronHowell in #52759
- chore(model_prices): add databricks-gemini-nano-banana-2-1 by @devin-ai-integration[bot] in #53027
- fix(cost): price a requested fireworks_ai router from its own row when Fireworks serves the base model by @kerry-berri in #52973
- test(e2e): move the Gemini Flash alias off the retired gemini-2.5-flash by @quietjules in #52823
- fix(count_tokens): count custom-named deployments with their base_model's tokenizer by @quietjules in #50734
- test(responses): await the WebSocket crash test without a deadline by @devin-ai-integration[bot] in #53035
- feat(db): route jwt provisioning writes through rust repositories by @yujonglee-berri in #52756
- feat(db): add jwt team members through the rust repository by @yujonglee-berri in #52766
- feat(db): insert audit logs through the rust repository by @yujonglee-berri in #52779
- fix(responses)!: reject previous_response_id on chat completions and Messages by @edvinsq in #53038
- fix(responses): preserve incomplete status in synthetic streams by @Bytechoreographer in #52489
- fix(cost-map): set openai gpt-6-rosalind long context and cache write prices to the pricing page values by @devin-ai-integration[bot] in #53050
- fix(gateway-auth): read the api-key header ahead of x-api-key, as the proxy does by @yujonglee-berri in #53026
- feat(guardrails): opt-in Bedrock scan of tool-call arguments and tool descriptions as text by @mateo-berri in #52999
- chore: drop narration comments added in the last day by @devin-ai-integration[bot] in #53053
- feat(budgets): support custom additional spend windows by @tin-berri in #52976
- refactor(redis): move per-request miss, settlement and shutdown-drain tracking from Python into Rust by @yujonglee-berri in #51384
- refactor(auth): name the keyspace in the JWT cache host calls by @yujonglee-berri in #52830
- refactor(proxy): move non-auth writers out of the auth cache by @yujonglee-berri in #51599
- feat(auth_cache): name each auth cache object with a typed key by @yujonglee-berri in #52831
- feat(customer-portal): share budget editors and app model access by @tin-berri in #52784
- feat(proxy): cache per-app customer authentication records by @tin-berri in #53056
- fix(customer-portal): simplify budget navigation by @tin-berri in #53066
- fix(proxy): bill a forwarded-credential clone's final timed-out attempt once by @kerry-berri in #53072
- fix(anthropic)!: send extra_body fields at the top level of the Messages request by @drweber in #48373
- chore(model_prices): remove databricks-gemini-nano-banana-2-1 until Databricks documents the endpoint by @devin-ai-integration[bot] in #53108
- fix(proxy): forward limit, order, after, before and include on GET /v1/responses/{id}/input_items by @quietjules in #51259
- chore(cost-map): rolling registry audit, Together/Perplexity/InferenceAPIs/AssemblyAI rows, xAI/Gemini/Anthropic metadata by @devin-ai-integration[bot] in #52912
- test(spend): expect soft budget duration in customer scopes by @devin-ai-integration[bot] in #53017
- test(perf): report unanswered RSS captures as probe incomplete, not worker death by @kerry-berri in #52867
- chore(docs): regenerate provider endpoints support for Together AI speech by @kerry-berri in #53124
- fix(ui): use one English name per admin UI concept by @yuneng-berri in #52968
- test(projects): assert both project cache eviction keys by @devin-ai-integration[bot] in #53129
- test(spend): supply the usage cache in budget reset tests by @devin-ai-integration[bot] in #53128
- fix(complexity-router): support optional classifier output limits by @mrinal-berri in #52979
- fix(cost): bill Together AI speech rows priced per character at that rate by @kerry-berri in #53130
- fix(proxy): rotate all database secrets with the master key by @yassin-berriai in #52998
- feat(proxy): send gateway model calls to Lens as traces when the Lens store is connected by @kerry-berri in #53042
- fix(cost): bill Perplexity embeddings at the cost Perplexity reports by @kerry-berri in #53125
- test: fix four flaky router and s3 log unit tests by @narmstrong-berri in #53034
- fix(openai): preserve omitted headers for chat and embeddings by @joshua-berri in #53132
- fix(streaming): preserve Azure model when bridge chunks omit it by @joshua-berri in #53126
- chore: bump litellm-enterprise 0.1.87 -> 0.1.88, litellm-proxy-extras 0.4.119 -> 0.4.120 by @yuneng-berri in #53143
- test(auth): hold the JWT mapping insert the Rust repository now sends by @yuneng-berri in #53144
- fix(proxy): preserve budget window edits during resets by @tin-berri in #53064
- feat(proxy): health check AI Hub models published through a wildcard by @ryan-crabbe-berri in #52875
- fix(gemini): support explicit text in embedding inputs by @joshua-berri in #53135
- test(lens): import TypedDict from typing_extensions so the Lens proxy tests pass on Python 3.10 by @kerry-berri in #53148
- fix(gemini): match tool results to calls split across assistant turns by @ryan-crabbe-berri in #52960
- test(perf): recalibrate the Postgres peak connection limit by @kerry-berri in #53121
- fix(proxy): tolerate unavailable psycopg during migrations by @joshua-berri in #53146
- fix(lens): do not send Lens's own analysis calls back to Lens as traces by @kerry-berri in #53158
- test(redis): exempt the state and hook cache classes from the key family check by @kerry-berri in #53157
- fix(mcp)!: make automatic gateway DCR registration opt-in by @shrey-berri in #53059
- fix(responses): send bridged system prompts too long for instructions as input by @ryan-crabbe-berri in #52964
- test(perf): remove Postgres peak connection measurement by @kerry-berri in #53154
- fix(ui): only offer teams where the caller can create keys in Create Key by @yassin-berriai in #53139
- fix(ui): keep auto-router simple setup when changing strategy by @moe-berri in #53054
- fix(proxy): skip model allow-lists on compliance check routes by @yuneng-berri in #53160
- chore(model_prices): add baseten bison and groundhog decision models by @devin-ai-integration[bot] in #53165
- chore(model_prices): add databricks gpt-5 and gpt-5-mini retirement dates by @devin-ai-integration[bot] in #53168
- fix(lens): update the embedded UI to the published release by @moe-berri in #53041
- chore(model_prices): mark xai/grok-build-0.1 as end of life on 2026-11-19 by @devin-ai-integration[bot] in #53167
- chore(cost-map): add runwayml/seedance2_5_flash by @devin-ai-integration[bot] in #53166
- chore(repo): remove committed Lens review screenshots by @moe-berri in #53169
- fix(bedrock): report Nova embedding input tokens from the Bedrock metered count by @kerry-berri in #53142
- test(lens): isolate exporter Prometheus counters by @devin-ai-integration[bot] in #53172
- fix(proxy): enforce file ownership on Gemini pass-through file get and delete by @kerry-berri in #53171
- feat(lens): treat Lens's own calls as ordinary calls labeled lens-signals and remove x-lens-internal by @kerry-berri in #53163
- fix(bedrock): keep the Nova embedding transform_response a plain forwarder by @kerry-berri in #53183
- test(aiohttp): verify streaming timeouts without wall-clock delays by @devin-ai-integration[bot] in #53189
- refactor(db): move the rust pool refusal probe into a RefusalWatch behind a FailFastPool by @narmstrong-berri in #52675
- test(realtime): place transcription sessions on both workers by @devin-ai-integration[bot] in #53141
- test(model-hub): compare matching pass-through rows by @devin-ai-integration[bot] in #53193
- feat(lens): send the calling agent's framework, task and session on gateway traces by @kerry-berri in #53186
- fix(decisions): accept OpenAI input_json content parts on /v1/decisions by @zoveki in #52805
- fix(proxy): stop a forged Authorization header from recording zero spend on an operator OAuth deployment by @kerry-berri in #53170
- fix(lens): start a new run when an agent with no session id repeats a prompt by @kerry-berri in #53194
- fix(lens): name gateway Codex runs codex-cli so they join Codex's own traces in Lens by @kerry-berri in #53197
- feat(lens): add /lens/login so a LiteLLM login can sign in to standalone Lens by @kerry-berri in #53190
- fix(budgets): preserve full elapsed intervals between resets by @tin-berri in #53151
- chore(ui): remove the decision models announcement banner by @kerry-berri in #53202
- fix(caching): preserve Responses objects on cache hits by @joshua-berri in #53188
- fix(proxy): echo the path model on nonstreaming chat responses when the body has none by @joshua-berri in #53192
- fix(exceptions): map OpenAI forbidden invalid requests to permission denied by @joshua-berri in #53177
- feat(proxy): delete the old username and password login page by @kerry-berri in #53205
- fix(lens): stop /lens/login and the dashboard login page redirecting to each other forever by @kerry-berri in #53212
- fix(cost): bill OpenAI token-priced speech on the length of the returned audio by @kerry-berri in #53150
- fix(proxy): close encoded and backslash paths that skip the Gemini pass-through file ownership check by @kerry-berri in #53210
- feat(lens): send the signed in user's name in the Lens login token by @kerry-berri in #53213
- fix(budgets): preserve calendar periods across DST by @tin-berri in #53065
- refactor(ui): remove unused validateToolDisplayName from MCP utils by @yuneng-berri in #52935
- refactor(ui): remove unused callback lookup helpers by @yuneng-berri in #52934
- fix(openai): relay param and plain message from OpenAI-compatible backend errors by @kenneth-berri in #53217
- feat(docker): install cryptography, grpcio and libpq from chainguard apks in the fips image by @yucheng-berri in #52997
- feat(cost-map): turn on Bedrock native structured output for GovCloud Claude rows by @kerry-berri in #53175
- fix(auth): return generic 401 for unknown JWT team by @moe-berri in #53044
- refactor(budgets): use one budget window model for keys, teams, users and customers by @tin-berri in #53218
- fix(proxy): honor the Cache-Control no-cache and no-store request headers by @Shanmukh-l33t in #52840
- fix(proxy): size budget reservation from route output cap fields by @xinglan163 in #51752
- fix(vertex_ai): fail a Gemini stream cut before its finish reason by @ryan-crabbe-berri in #53187
- feat(callbacks): spend log contract fields and native flush at shutdown by @narmstrong-berri in #52151
- feat(budgets): run the ledger's post-call spend counter step in rust by @narmstrong-berri in #52166
- feat(spend-logs): rust spend log row builder by @narmstrong-berri in #52161
- feat(spend-logs): rust spend log ledger writer by @narmstrong-berri in #52163
- feat(spend-logs): rust spend log callback by @narmstrong-berri in #52182
- test(tags): give the tag update fake a database transaction by @devin-ai-integration[bot] in #53230
- test(s3): prove unlisted upload statuses are not retried from observed reports by @devin-ai-integration[bot] in #53229
- fix(proxy): name configured customer headers in authorization errors by @moe-berri in #53043
- test(responses): wait on events instead of a one second deadline for closed socket rows by @devin-ai-integration[bot] in #53228
- refactor(auth_cache): name the customer app cache families as AuthObjects by @yujonglee-berri in #53088
- feat(guardrails): hold streamed tool calls for the verdict and log prompts blocked with a reply by @srik-straiker in #51872
- fix(gemini): preserve distinct parallel tool calls with identical arguments by @joshua-berri in #53233
- test(integration): stop the Bedrock invoke outage test from assuming every cut-off stream is billed in full by @yucheng-berri in #53239
- fix(responses): drop unsupported o-series temperature when requested by @joshua-berri in #53234
- fix(caching): preserve stored completion finish reasons by @joshua-berri in #53236
- feat(realtime): serve Azure GPT-Live sessions on /v1/live/sessions by @moe-berri in #53238
- fix(responses): connect the Responses WebSocket to a plain http api_base by @kenneth-berri in #53240
- fix(gemini realtime): report barge-in as cancelled response.done by @mrinal-berri in #52596
- test(translation): heal stale Messages fixtures for bridged system input and unsigned Claude thinking by @yuneng-berri in #53243
- fix(lens): keep the cost of a request too large to export whole by @kerry-berri in #53268
- fix(mcp): preserve concurrent OAuth scope refusals by @shrey-berri in #52809
- fix(router): preserve team order and weighted failover across endpoints by @joshua-berri in #53264
- fix(lens): send cost records to Lens without the conversation by @kerry-berri in #53274
- fix(transcription): log the clip duration as usage when the provider sends none by @kenneth-berri in #53256
- fix(spend): retain billed failure accounting when error logs are disabled by @joshua-berri in #53271
- feat(guardrails): reject requests that name an unregistered guardrail when reject_unregistered_guardrails is on by @mrinal-berri in #53214
- fix(cost): bill speech at a deployment's custom per-character price by @kenneth-berri in #53265
- fix(spend): read budget windows from the auth cache under the auth objects key family by @mateo-berri in #53287
- fix(lint): clear the code-quality failures on main by @narmstrong-berri in #53237
- test(e2e): run codex in fast mode under the real model name by @ryan-crabbe-berri in #49899
- chore(deps): bump zip from 2.4.2 to 9.0.5 in /litellm/rust by @dependabot[bot] in #51354
- chore(lens): update the embedded UI to Lens 0.2.0 by @kerry-berri in #53281
- fix(ui): regenerate the dashboard API types for reject_unregistered_guardrails by @mateo-berri in #53289
- test(perf): gate large-body case on response time and CPU with a loose client p50 ceiling by @kerry-berri in #53155
- test(auth): expect the CLI sign-in redirect by @devin-ai-integration[bot] in #53294
- test(management): account for the ledger writer reserve by @devin-ai-integration[bot] in #53293
- fix(messages): preserve thread refusal details by @devin-ai-integration[bot] in #53292
- test(mcp): give the pin test's proxy_server stub the general settings reader by @mateo-berri in #53295
- chore(cost-map): rolling registry audit 2026-11-20, capability metadata, Azure AI GPT-5.6 and OpenRouter DeepSeek V4 pricing, Vertex Gemini image fields by @devin-ai-integration[bot] in #53149
- chore(model_prices): move fireworks_ai inkling deprecation date to 2026-11-20 by @devin-ai-integration[bot] in #53303
- refactor(typing): clear 1146 basedpyright Any errors across 365 files by @mateo-berri in #52768
- fix(proxy): read audit update rows stored before #50929 in GET /audit by @kerry-berri in #53305
- fix(anthropic): forward diagnostics on /v1/messages by @ryan-crabbe-berri in #52969
- refactor(messages): add a conformance table and one layout, naming and order for messages configs by @narmstrong-berri in #53199
- refactor(messages): split the messages config into four parts and build the request in one pipeline by @narmstrong-berri in #53203
- refactor(messages): write anthropic-family configs as anthropic plus named host edits by @narmstrong-berri in #53204
- feat(provider)!: add ResetData and stop sending OPENAI_API_KEY to keyless JSON-configured providers by @TheRealKraken in #50819
- test(files): measure only upload path allocations in the large upload rewrite test by @devin-ai-integration[bot] in #53318
- test(spend): assert ledger callback counts apart from wire command order by @devin-ai-integration[bot] in #53317
- test(bedrock): stub realtime signing in the guardrail opt out test by @devin-ai-integration[bot] in #53319
- feat(gateway): serve inference through router-core with the host glue it needs by @yujonglee-berri in #52274
- refactor(auth_cache): type auth cache keys and keep the cache always native by @yujonglee-berri in #51130
- refactor(auth_cache_invalidation): move the invalidation channel to the native client by @yujonglee-berri in #51135
- refactor(auth-store): shape the cached virtual key in Rust and load it through one Python path by @yujonglee-berri in #53325
- feat(gateway): model restamping in rust gateway by @yujonglee-berri in #52428
- feat(gateway): answer a stream the provider fails before any content with a retry or a fallback by @yujonglee-berri in #52466
- fix(vertex_ai): fail Gemini malformed replies with no output so retries and fallbacks run by @ryan-crabbe-berri in #53231
New Contributors
- @Aditya-Pathanjali made their first contribution in #50929
- @edvinsq made their first contribution in #53038
- @drweber made their first contribution in #48373
- @Shanmukh-l33t made their first contribution in #52840
- @TheRealKraken made their first contribution in #50819
Full Changelog: v1.112.0-dev.1...v1.112.0-dev.2