v1.96.0

Verify Docker Image Signature

All LiteLLM Docker images are signed with cosign. Every release is signed with the same key introduced in commit 0112e53.

Verify using the pinned commit hash (recommended):

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
  ghcr.io/berriai/litellm:v1.96.0

Verify using the release tag (convenience):

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/v1.96.0/cosign.pub \
  ghcr.io/berriai/litellm:v1.96.0

Expected output:

The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key

---

What's Changed

• feat(prometheus): add service_tier label to latency and spend metrics by @yassin-berriai in #34966
• fix(aiohttp): keep keep-alive connector config when a session is rebuilt by @yassin-berriai in #34962
• fix(helm): pin bundled postgres and redis to the bitnamilegacy images by @yassin-berriai in #34963
• fix(vertex): decide rawPredict passthrough streaming from the request body by @tin-berri in #34672
• fix(anthropic-adapter): translate stop_sequences and disabled thinking for non-Claude targets by @tin-berri in #34589
• ci: publish a generated JSON schema for model_prices_and_context_window.json by @mateo-berri in #34816
• fix(jwt_auth): allow /v1/messages for JWT teams by default by @devin-ai-integration[bot] in #34222
• fix(proxy): skip team model aliases that point at deleted deployments by @mateo-berri in #34993
• fix(proxy): report when a model write does not survive the post-write reload by @tin-berri in #34861
• fix(proxy): reject model writes that corrupt an auto-router pseudo-model by @tin-berri in #34151
• fix(router_strategy): serialize latency for non-chat responses in lowest-latency routing by @mihidumh in #33290
• fix(logging): preserve callback order in get_combined_callback_list by @Harsh23Kashyap in #33008
• fix(router): honor explicit priority=0 in acompletion by @Harsh23Kashyap in #33214
• fix(router_strategy): serialize latency for non-chat responses in lowest-latency routing by @tin-berri in #35048
• fix(anthropic-adapter): open the first content block with the real upstream type so reasoning-first streams start with thinking by @Napuh in #34433
• test(e2e): poll MCP tools across multi-worker lag by @mubashir1osmani in #35047
• fix(scim): stop provisioning nested group ids as internal users by @ryan-crabbe-berri in #34997
• chore(typing): clear basedpyright Any errors in proxy management endpoints by @mateo-berri in #35074
• fix(ui): size Object Permissions card grid by container width by @ryan-crabbe-berri in #35019
• fix(proxy): allow /key/update to identify the key by key_alias by @ryan-crabbe-berri in #34851
• feat(ui): split failed requests into their own series on the cache dashboard by @ryan-crabbe-berri in #34862
• feat(ui): shareable log links via log_id query param on the logs page by @ryan-crabbe-berri in #34879
• fix(ui): show public model names in usage breakdowns by @mateo-berri in #35107
• fix(ui): point the navbar and sidebar logos at the dashboard home route by @yuneng-berri in #35041
• fix(proxy): stop serving stale team model allowlist after /team/update by @mateo-berri in #34266
• feat(ui): deep link team detail page via ?team= query param by @ryan-crabbe-berri in #35112
• feat(ui): deep link organization detail page via ?org= query param by @ryan-crabbe-berri in #35117
• fix(proxy): resolve named credentials on provider-only batch and files calls by @mateo-berri in #35028
• fix(router): drop duplicate Mapping import that fails ruff F811 by @ryan-crabbe-berri in #35122
• refactor(rust): make litellm-core the callable messages() SDK; drop the ai-gateway handler by @devin-ai-integration[bot] in #35044
• fix(otel): make OTLP export work against Grafana Cloud by @yassin-berriai in #35060
• test(e2e): remove the Presidio guardrail suite by @mubashir1osmani in #35129
• feat(dashscope): add qwen3.7-plus and qwen3.7-max to the model cost map by @devin-ai-integration[bot] in #35123
• feat(ui): link organization teams to their team detail pages by @ryan-crabbe-berri in #35120
• fix(tests): assert Content variants are identified by type, not by the discriminator keyword by @yassin-berriai in #35161
• fix(mcp): never write discovery results to the row, heal already-stamped rows, and retry failed discovery with backoff by @devin-ai-integration[bot] in #34990
• fix(vertex_ai): honor per-model gcs_bucket_name on managed-file read path by @devin-ai-integration[bot] in #34847
• feat(cli): read base_url from persistent config file by @ryan-crabbe-berri in #35015
• fix(auth): resolve managed batch/file deployment model_id to model name for team access checks by @devin-ai-integration[bot] in #32587
• fix(vertex_ai): forward function_call id on Vertex Gemini 3+ tool turns by @ljogeiger in #34603
• build(makefile): give local basedpyright runs the node heap CI uses by @mateo-berri in #35173
• fix(complexity_router): capture the classifier request body in spend logs by @tin-berri in #35164
• test(managed-files): lock in store_unified_file_id idempotency on batch retrieve by @devin-ai-integration[bot] in #32601
• fix(vertex_ai): skip context caching when the cached block ends on a model turn by @mateo-berri in #35172
• refactor(bedrock): remove the dead BedrockLLM invoke code path by @mateo-berri in #35188
• fix(pricing): correct gpt-5.4-mini and gpt-5.4-nano token limits by @mateo-berri in #35182
• feat(cookbook): add a Grafana dashboard for the OTel GenAI metrics by @yassin-berriai in #35159
• chore(typing): clear basedpyright Any errors in proxy auth, repositories, and openai transforms by @mateo-berri in #35240
• fix(batches): calculate cost and usage for completed Vertex AI batches by @mateo-berri in #35186
• feat(ui): give auto-routers their own tab on Models + Endpoints by @tin-berri in #35009
• refactor(batches): aggregate batch output cost, usage, and models in a single pass by @mateo-berri in #35205
• feat(router): record why the auto-router picked a tier in spend logs by @tin-berri in #35016
• fix(otel): cap tool-definition attributes so they cannot evict gen_ai.* from the LLM span by @yassin-berriai in #34828
• feat(mcp): enforce per-user MCP tool-call entitlements in the auth module by @yassin-berriai in #35146
• fix(otel): record the GenAI duration metric on failed requests by @yassin-berriai in #35152
• chore: bump litellm 1.95.0 -> 1.96.0 by @yuneng-berri in #35254
• fix(pricing): adjust gpt-5.6-terra and gpt-5.6-luna prices according to OpenAI's latest article by @lihugang in #35258
• revert(proxy)!: stop enforcing user budget on team keys by @yuneng-berri in #35271
• fix(otel): label retrieval and agent metrics correctly and emit gen_ai.provider.name by @yassin-berriai in #35151
• test(e2e): align budget e2e with the team-key budget hierarchy by @yuneng-berri in #35276
• feat(mcp): scan and mask MCP tool results via post_mcp_call guardrails by @yassin-berriai in #35155
• docs(claude): require 15-25 word human-readable replies to AI PR review bots by @devin-ai-integration[bot] in #35266
• fix(redis): stop an unreachable Redis from blocking every request by @yassin-berriai in #35273
• fix(rate-limits): keep the v3 limiter out of provider-facing metadata on responses routes by @devin-ai-integration[bot] in #35207
• docs(pr-template): require e2e proof on all three LLM endpoints when applicable by @mateo-berri in #35280
• feat(mcp): manual authorization-code delivery for headless MCP clients by @tin-berri in #34848
• feat(db): opt-in REPLICA IDENTITY FULL after prisma migrations by @yassin-berriai in #35267
• fix(ui): stop clamping the budgets Budget ID column at 15 characters by @yuneng-berri in #35268
• fix(proxy): only enforce budgets on routes that can spend by @ryan-crabbe-berri in #35274
• chore(ci): promote internal staging to main by @yuneng-berri in #35285
• fix(caching): stamp provider on embedding cache-hit spend logs by @devin-ai-integration[bot] in #35282
• fix(proxy): run post_call guardrails on /v1/messages streaming via unified guardrail translation by @mateo-berri in #35260
• fix(fireworks_ai): correct Kimi K2.5/K2.6/K2.7 max output token limits by @devin-ai-integration[bot] in #35174
• feat(mcp)!: extend keyless gateway OAuth flow to per-server MCP URL paths by @tin-berri in #34856
• fix(proxy): request stream usage upstream by default and strip it from client streams by @mateo-berri in #35290
• refactor(rate-limits): move the v3 limiter per-request stash off request metadata onto a ContextVar by @mateo-berri in #35278
• test(e2e): skip the batch rate-limiter spend-row test pending LIT-5027 by @ryan-crabbe-berri in #35301
• fix(guardrails/headroom): stop compressing the turn the model must act on by @tin-berri in #35294
• fix(ui): let the internal user and org forms save sub-cent budgets by @ryan-crabbe-berri in #35302
• fix(guardrails): serve config guardrails from list and info endpoints without a DB and make their ids stable by @mateo-berri in #35259
• fix(anthropic): split mixed stream chunks by payload kind by @yucheng-berri in #35289
• fix: give ComplexityRouter LLM classifier prior-turn context (LIT-4981) by @tin-berri in #35185
• feat(spend-logs): mark the auto-router's own classifier calls by @tin-berri in #35300
• fix(logging): bind litellm_metadata by reference in function_setup so guardrail info reaches spend logs by @yucheng-berri in #35292
• feat(s3): support SSE-KMS encryption params on both S3 logging paths by @yucheng-berri in #35291
• fix(policy_engine): preserve config-defined policies across DB sync and expose them via list APIs by @mateo-berri in #35263
• test: fix order-dependent flake in passthrough guardrail call-type test by @mateo-berri in #35317
• fix(pricing): correct gpt-5.6 prices for openai, bedrock, and flex long context by @mubashir1osmani in #35270
• fix(cost): bill the fast service tier at the priority rate by @devin-ai-integration[bot] in #35320
• fix(responses): map all documented in-stream error codes to real HTTP statuses by @mateo-berri in #35307
• test(pricing): cover gpt-5.6 cache-cost plumbing and bedrock_mantle responses billing by @mateo-berri in #35324
• chore: promote staging to main by @mateo-berri in #35328
• fix(mcp): annotate connected-app reachability on the gateway connect page by @tin-berri in #34867
• fix(mcp): recover the tool-name prefix boundary from registered prefixes by @tin-berri in #34673
• fix(e2e): exclude skipped tests from coverage-registry numerator by @yuneng-berri in #35327
• feat(ui): expose classifier context window fields on Auto-Router screens (LIT-5036) by @tin-berri in #35315
• fix(mcp): deny MCP access when a named entitlement cannot be read by @yassin-berriai in #35160
• feat(proxy): add generic list handler for /management/v1 by @yuneng-berri in #35308
• fix(aiohttp): dispose recycled client sessions deterministically by @yassin-berriai in #33428
• test(e2e): skip the three Datadog MCP tool-call tests pending LIT-5052 by @ryan-crabbe-berri in #35380
• feat(mcp): source the ID-JAG subject from the user's stored SSO assertion by @yassin-berriai in #35147
• test(e2e): skip the throughput SLO load test pending LIT-5054 by @ryan-crabbe-berri in #35381
• fix(guardrails): run bedrock guardrail on MCP tool calls in during_mcp_call mode by @yassin-berriai in #35149
• fix(tool-management): drop unsupported prisma select kwarg from team lookup by @yuneng-berri in #35293
• feat(proxy): add GET /management/v1/budgets by @yuneng-berri in #35310
• fix(gemini): do not send duplicate thoughtSignature copies to Gemini by @mgeorgaklis in #35004
• feat(ui): show which log rows are the auto-router's own classifier calls by @tin-berri in #35304
• fix(ui): nest source object in Claude Code marketplace settings snippet by @devin-ai-integration[bot] in #35322
• fix(type-discipline): exempt values frozen in place by tuple/frozenset/MappingProxyType from LIT002 by @mateo-berri in #35325
• feat(ui): add sorting, filtering and search to the budgets page by @yuneng-berri in #35309
• fix(ui): keep the session view open when selecting a log inside it by @ryan-crabbe-berri in #35399
• fix(mcp): enforce tool entitlements on every MCP tool dispatch path by @yassin-berriai in #35156
• fix(deps): raise aiohttp floor to 3.14.2 to clear pooled-connection timeouts by @yuneng-berri in #35337
• fix(tag-management): drop unsupported prisma select kwarg from key lookup by @yuneng-berri in #35288
• fix(proxy): stop model writes 500ing on another pod's delete by @ryan-crabbe-berri in #35400
• fix(ui): show pass through route selections and match team id substrings in team search by @devin-ai-integration[bot] in #35319
• chore(typing): replace Any kwargs unpacking with validated model parsing by @mateo-berri in #35397
• feat(proxy)!: gate all mock testing request params behind a single config flag by @yuneng-berri in #35423
• chore(typing): clear 2.4k basedpyright errors across 15 Any hotspot files by @mateo-berri in #35452
• fix(proxy): align team member add with existing user provisioning rules by @yuneng-berri in #35435
• feat(prometheus): add global exclude_metrics and exclude_labels options by @devin-ai-integration[bot] in #34201
• feat(proxy): let AI API keys read /model/info by @yuneng-berri in #35473
• chore(proxy): remove duplicate Sequence import in team endpoints by @yuneng-berri in #35479
• fix(a2a): keep config-defined agents registered and accept the documented agents: key by @yassin-berriai in #35163
• ci: run unit tests on pushes to main and litellm_internal_staging by @yuneng-berri in #35475
• chore: bump litellm-enterprise 0.1.52 -> 0.1.53 by @yuneng-berri in #35476
• chore(ui): rebuild committed Admin UI bundle by @yuneng-berri in #35484
• fix(rate-limit): enforce token limits when the pre-call increment is zero by @shivamrawat1 in #35422
• fix(helm): render pod-level securityContext on the migration Job by @yassin-berriai in #35482
• fix(router): honor request-level num_retries over a deployment's litellm_params value by @yassin-berriai in #35483
• fix(spend): bound each spend-log write statement by payload bytes by @yassin-berriai in #34956
• feat(proxy): bound DB statement and lock time via general_settings by @yassin-berriai in #35496
• fix(bedrock): cache AssumeRole credentials per attributed identity by @yassin-berriai in #35467
• feat(complexity_router): let the classifier see assistant turns and rate what a short reply approves by @tin-berri in #35471
• feat(proxy): push config sync to pods via redis pub/sub by @mateo-berri in #35436
• test(e2e): derive the throughput SLO per replica and surface locust's error breakdown by @yassin-berriai in #35494
• feat(helm): add pod-hardening and migration-Job knobs to the componentized chart by @yassin-berriai in #35489
• fix(docker): bake prisma offline in the componentized migrations image by @yassin-berriai in #35485
• fix(batches): encode public model group on background-created output file ids by @devin-ai-integration[bot] in #35406
• fix(componentized): honor USE_DDTRACE in the gateway and backend deployments by @yassin-berriai in #35490
• fix(helm): give gateway and backend probes an explicit timeoutSeconds by @yassin-berriai in #35497
• fix(ci): let the E2E proxy accept the mock testing params its suite sends by @yuneng-berri in #35511
• test(proxy): separate the member_add permission gate from the provisioning gate by @yuneng-berri in #35507
• test(logging): pin routing_decision and internal_call_origin in the gcs pubsub spend log fixture by @yuneng-berri in #35506
• feat(ui): expose the assistant-turn classifier context switch on Auto-Router screens by @tin-berri in #35500
• test(proxy): assert _delete_deployment's still-desired id set instead of a delete count by @yuneng-berri in #35505
• refactor(complexity_router): drop the tier-rubric override, close the rubric on the window it was given by @tin-berri in #35504
• fix(deps): move pydantic-settings into the base dependencies by @yuneng-berri in #35518
• refactor(lint): apply every safe ruff autofix and zero 28 strict-rule budgets by @mateo-berri in #35495
• build(makefile): swap npm ci for npm install so bootstrap no-ops on unchanged ui deps by @mateo-berri in #35509
• fix(team-callbacks): report API-registered callbacks from GET /team/{team_id}/callback by @yucheng-berri in #35512
• fix(ui): land general login on the keys dashboard, send MCP consent to /ui/connect by @tin-berri in #35523
• build(makefile): run bootstrap before pre-commit lint by @mateo-berri in #35547
• chore: rebuild Admin UI bundle by @yuneng-berri in #35550
• test(e2e): skip the strict-priority and throughput SLO tests pending LIT-5118 / LIT-5119 by @ryan-crabbe-berri in #35575
• test(e2e): poll key spend to a deadline in budget reset advances tests by @ryan-crabbe-berri in #35572
• chore(ci): promote internal staging to main by @yuneng-berri in #35501
• chore(release): backport #35835, #35844 to rc/1.96.0 and refresh dependency pins by @yuneng-berri in #36294
• chore: rebuild Admin UI bundle for rc/1.96.0 by @yuneng-berri in #36320

New Contributors

• @Harsh23Kashyap made their first contribution in #33008
• @ljogeiger made their first contribution in #34603
• @lihugang made their first contribution in #35258

Full Changelog: v1.95.0...v1.96.0