v1.101.0-rc.1

Verify Docker Image Signature

All LiteLLM Docker images are signed with cosign. Every release is signed with the same key introduced in commit 0112e53.

Verify using the pinned commit hash (recommended):

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
  ghcr.io/berriai/litellm:v1.101.0-rc.1

Verify using the release tag (convenience):

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/v1.101.0-rc.1/cosign.pub \
  ghcr.io/berriai/litellm:v1.101.0-rc.1

Expected output:

The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key

---

What's Changed

• fix(spend_tracking): key /v1/messages spend rows on the msg_ id the client received by @mateo-berri in #39511
• ci(rust): build and test the ai-gateway server feature by @mateo-berri in #39493
• ci(ui): run the UI build check through the image's ui-builder stage by @mateo-berri in #39496
• fix(proxy): parse numeric multipart fields on /v1/images/edits back into numbers by @mateo-berri in #39510
• fix(guardrails): remove the module-global translation mapping that leaked between tests by @mateo-berri in #39543
• feat(azure_ai): add grok-4.6 to the model cost map by @mateo-berri in #39426
• fix: attach vector store search_results when a guardrail is registered by @mateo-berri in #38984
• fix(proxy): stop putting the literal string "None" in error payloads by @mateo-berri in #39521
• fix(router): keep retry breadcrumbs per request and out of the request snapshot by @mateo-berri in #39491
• fix(vector-stores): survive a failing vector store search in the chat completions hook by @mateo-berri in #39495
• fix(utils): redact credential kwargs from the set_verbose request line by @mateo-berri in #39526
• fix(bedrock): skip the SigV4 credential chain when a bearer token is configured by @mateo-berri in #39411
• fix(proxy-extras): kill the whole Prisma process group when a command times out by @mateo-berri in #39466
• fix(rag): forward the managed vector store's params to the search call by @mateo-berri in #39452
• fix(utils): redact credentials nested in extra_body on the verbose optional-params line by @mateo-berri in #39538
• fix(containers): pass upstream error status through and forward list pagination params by @mateo-berri in #39464
• fix(anthropic_messages): key bridged streaming spend rows on the streamed msg_ id by @mateo-berri in #39541
• fix(helm): render ingress-nginx compatible path types via ingress.controller by @mateo-berri in #39465
• fix(cache): use sync Redis batch reads by @yucheng-berri in #39358
• fix(guardrails): rebuild the serving worker guardrail on PUT instead of patching it in place by @mateo-berri in #39243
• test(team-race): wait on pg_locks instead of a fixed sleep by @yuneng-berri in #39611
• fix(agents): hide agents from non-admins who were never granted them by @devin-ai-integration[bot] in #39636
• fix(team_endpoints): stop partial /team/update from wiping team metadata by @devin-ai-integration[bot] in #36328
• test(router): cover configured mode lookup by @moe-berri in #39634
• fix(responses): encrypt the response id on every streamed event by @mateo-berri in #39534
• fix(mcp): resolve OAuth broker endpoints by server_id with IP access checks by @devin-ai-integration[bot] in #39432
• feat(proxy): enforce team isolation for provider-format batch ids and output files by @yuneng-berri in #33536
• fix(mcp): strip inbound auth scheme case-insensitively before token exchange by @devin-ai-integration[bot] in #39346
• fix(mcp): normalize a schemed authentication_token on the v2 and OpenAPI static paths by @devin-ai-integration[bot] in #39345
• fix(docker): match USE_DDTRACE case-insensitively and route build_from_pip through prod_entrypoint.sh by @devin-ai-integration[bot] in #39344
• perf(auth): skip object permission DB lookup when no vector stores requested by @devin-ai-integration[bot] in #39347
• fix(caching): don't trip redis circuit breaker on short timeout bursts by @devin-ai-integration[bot] in #38999
• fix(access_groups): derive attached teams from the team table and reject unknown team ids by @ryan-crabbe-berri in #39218
• test(router): drop duplicate get_configured_mode test failing ruff F811 by @mateo-berri in #39659
• feat(cost): honor off_peak_pricing reasoning and cache-creation rates by @mateo-berri in #39635
• fix(openai): mint workload identity tokens for PrivateLink and regional api.openai.com hosts by @mateo-berri in #39652
• feat(ui): find rows by a pasted ID on keys, agents, memory, audit, and request logs by @ryan-crabbe-berri in #39661
• test(proxy-extras): fake run_prisma instead of subprocess.run in the migrate deploy harness by @yuneng-berri in #39669
• fix(scim): default-team fallback on create and keep memberships when PUT /Users has no groups by @devin-ai-integration[bot] in #39623
• fix(router): count tools and Anthropic system prompt in context-window pre-call check by @devin-ai-integration[bot] in #39663
• test(proxy-extras): repoint the migrate-deploy harness at the run_prisma seam by @mubashir1osmani in #39673
• perf(spend): group /spend/logs summary by day in Postgres instead of per-row Prisma group_by by @devin-ai-integration[bot] in #39351
• fix(proxy): apply default_vertex_config location before building the Vertex passthrough base URL by @mateo-berri in #39662
• fix(anthropic_endpoints): return Anthropic type:error envelope for /v1/messages errors by @mateo-berri in #39037
• feat(ui): configure auto-router session affinity TTL by @tin-berri in #39679
• perf(mcp): cache SSO identity assertion reads on the ID-JAG path by @devin-ai-integration[bot] in #39348
• test(e2e): match the Internal Users search placeholder shipped by #39604 by @mateo-berri in #39678
• fix(ui): scroll admin table rows inside the table instead of the page by @ryan-crabbe-berri in #39684
• fix(router): evict stale global pattern_router entries on upsert/delete by @devin-ai-integration[bot] in #39664
• fix(caching): keep a node timeout from forcing a cluster-wide topology reinit on redis-py 8.x by @devin-ai-integration[bot] in #39349
• fix(logging): blocked requests no longer report guardrail_status=success in multi-guardrail configs by @yucheng-berri in #39596
• fix(snowflake): normalize Cortex Claude request shapes by @tin-berri in #39453
• feat(proxy): per-worker admission control that rejects excess requests with 503 by @devin-ai-integration[bot] in #39352
• fix(proxy): emit SSE keepalives on queue, rag, azure passthrough, usage chat and policy enrich streams by @devin-ai-integration[bot] in #39273
• fix(azure): restrict the storage credential chain to deployment identities by @yucheng-berri in #39637
• fix(model_checks): drop wildcard routes like bedrock/* from /v1/models by @devin-ai-integration[bot] in #31731
• fix(mcp): pre-flight the ID-JAG credential at the transport edge by @yassin-berriai in #35392
• feat(caching): add semantic_cache_scope to isolate semantic cache hits per end user by @devin-ai-integration[bot] in #39590
• feat(ui): add one-click Auto Router setup by @moe-berri in #39693
• fix(ci): pin setup-uv to v10.0.1 by @yujonglee-berri in #39111
• refactor(tests): restructure rust python harness around strategy definitions by @yujonglee-berri in #39628
• test(ocr): complete Rust unit test parity by @yujonglee-berri in #39689
• feat: page the public model hub table off /public/v1/model_hub, keeping every filter by @yuneng-berri in #39691
• refactor(rust): extract config crate by @yujonglee-berri in #39706
• feat(python): rename Rust rollout API by @yujonglee-berri in #39704
• fix: restore Python compatibility and test 3.10 through 3.14 by @yujonglee-berri in #39399
• fix(spend-tracking): keep batch spend keys joinable after v1.99 provenance gate by @mateo-berri in #39568
• test: repair four chronically failing CI tests by @yuneng-berri in #39770
• feat(team): report per-user spend within a team for JWT traffic by @devin-ai-integration[bot] in #39771
• feat(router): add classifier circuit breaker by @moe-berri in #39701
• ci: report every failing test in a job instead of stopping at the first by @yuneng-berri in #39772
• test(caching): drive the redis stall burst off the clock, not asyncio.wait_for by @yuneng-berri in #39773
• fix(ui): clamp server-paginated DataTable page index when rowCount shrinks by @ryan-crabbe-berri in #39776
• feat(mcp): use x-mcp-<access_group>-* headers as default upstream credentials for group members by @devin-ai-integration[bot] in #39717
• fix(ui): paginate per-user usage with the shared server-side DataTable footer by @devin-ai-integration[bot] in #39682
• fix(proxy): log mid-stream /v1/messages failures as failures with partial usage by @mateo-berri in #39589
• fix(cost): honor off_peak_pricing in the fireworks_ai and perplexity cost calculators by @mateo-berri in #39632
• fix(organization): clear org budget limits when PATCH /organization/update sends null by @devin-ai-integration[bot] in #39670
• fix(organization): reject negative limits and unparseable budget_duration on PATCH /v2/organization by @ryan-crabbe-berri in #39793
• feat(organization): expose PATCH /v2/organization/{organization_id} in the OpenAPI schema by @ryan-crabbe-berri in #39794
• fix(auto-router): route 1M complex tier to GPT Sol by @tin-berri in #39797
• fix(bedrock_mantle): anchor MANTLE_HOST_RE so custom Mantle hosts are honored by @mateo-berri in #39361
• fix(health): probe test_connection with the credential the request names by @yuneng-berri in #39801
• fix(router): bound auto-router classifier latency by @moe-berri in #39696
• fix(ui): accept any routing group name the backend accepts by @ryan-crabbe-berri in #39807
• fix(model_prices): verified registry audit, Databricks Sep-2026 catalog, realtime image pricing, deprecation dates by @devin-ai-integration[bot] in #39388
• fix(jwt): invalidate JWT key mapping cache on /key/regenerate by @ryan-crabbe-berri in #39808
• fix(complexity_router): fall back to a live peer when the decided tier model is fully cooled down by @tin-berri in #39675
• feat(guardrails): roll up Bedrock guardrail cost per usage counter by @ryan-crabbe-berri in #39196
• fix(auto_router): derive tier definitions in prompt editor by @tin-berri in #39688
• fix(proxy): recognize opencode's bare x-session-id header for session affinity by @devin-ai-integration[bot] in #39802
• feat(cli): sync OpenCode models from /v1/models in lite opencode by @devin-ai-integration[bot] in #39789
• fix(spend-tracking): keep internal service-account key names readable in spend logs by @devin-ai-integration[bot] in #39572
• fix(anthropic): never carry cache_control on translated thinking blocks by @moe-berri in #39815
• fix(responses/mcp): keep follow-up calls stateless when store=false by @devin-ai-integration[bot] in #36575
• fix(router): resolve retry_policy by exception hierarchy, add ServiceUnavailableErrorRetries and DefaultRetries by @shivamrawat1 in #35853
• fix(headroom): bound the /v1/compress and /v1/retrieve calls with a timeout by @yucheng-berri in #39527
• feat(vector_stores): add a MongoDB vector store provider for Atlas and self-managed deployments by @yuneng-berri in #39811
• fix(fireworks_ai): resolve tool_choice and reasoning support for short model names by @devin-ai-integration[bot] in #39763
• fix(datadog_llm_obs): keep the guardrail audit record under message redaction by @yucheng-berri in #39702
• fix(proxy): invalidate end-user spend counter and cache on budget reset (#39726) by @amasen02 in #39729
• fix(proxy): strip every TypedDict qualifier before numeric form-field detection by @yuneng-berri in #39780
• fix(bedrock): stop sending toolConfig tool definitions to guardrails on passthrough converse by @yucheng-berri in #39281
• feat(shadow_eval): judge tool-call turns instead of dropping or erroring on them by @moe-berri in #39818
• feat(router): auto-escalate stalled complexity-router tasks by @moe-berri in #39809
• fix(team_endpoints): let member_delete clear a team left on the user row by @devin-ai-integration[bot] in #38703
• feat(cost-map): add azure/gpt-6-astra and azure/us/gpt-6-astra Foundry pricing by @mateo-berri in #39827
• feat(complexity_router): let the LLM classifier see request images by @tin-berri in #39825
• feat(access-groups): resolve resource names on access group responses by @ryan-crabbe-berri in #39822
• fix(datadog_llm_obs): keep guardrail_cost_by_unit on redacted spans by @mateo-berri in #39848
• test(store_model_in_db): assert the 400 contract in the unknown-model spend log test by @mateo-berri in #39842
• feat(cli): add lite debug claude session report and /debug-lite slash command by @devin-ai-integration[bot] in #39435
• test: deflake JWT tamper, fuzzy picker, tag routing, liveliness, redis stall burst, and pre-commit interrupt tests by @devin-ai-integration[bot] in #39306
• feat(otel): stamp litellm.request.route on the LLM call span by @devin-ai-integration[bot] in #39698
• feat(shadow_eval): scope a job to model groups, ANDed with its key, team, and user targets by @tin-berri in #39828
• test(e2e/batches): assert Bedrock batch cancel and list in the lifecycle by @mateo-berri in #39847
• refactor: clear fresh tech debt from the last 24 hours (2026-09-03, 2026-09-04) by @devin-ai-integration[bot] in #39518
• refactor(typing): cut 1,397 Any errors across 183 backend files by @mateo-berri in #39461
• fix(responses): decode JSON-string tool schemas before sending to the provider by @tin-berri in #39844
• feat(helm): render nodeSelector, tolerations, and affinity on the componentized chart migrations Job by @mateo-berri in #39843
• fix(mcp): let config.yaml MCP servers pin server_id by @yucheng-berri in #39286
• feat(dashboard): configure classifier vision input by @tin-berri in #39840
• fix(batches): register ownership for every batch create path by @yuneng-berri in #39810
• fix(proxy): gate the OpenAI websocket passthrough behind an explicit opt-in by @mateo-berri in #39841
• fix(ui): bring the inline-object lint budget back under its ceiling by @mateo-berri in #39856
• fix(realtime): relay the upstream websocket close to the client instead of hanging by @mateo-berri in #39851
• feat(router): meter auto-router tier and prompt customization against the auto_router license feature by @tin-berri in #39674
• fix(shadow_eval): size the judge output cap for a judge that reasons by @moe-berri in #39817
• feat(organization): expose PATCH /v2/organization/{organization_id} in the OpenAPI spec by @devin-ai-integration[bot] in #39672
• fix(ui): make Admin UI table pagination honor the selected page size by @devin-ai-integration[bot] in #39680
• chore: bump litellm-enterprise 0.1.64 -> 0.1.65, litellm-proxy-extras 0.4.93 -> 0.4.94 by @yuneng-berri in #39912
• test(e2e): repair the wildcard readiness probe and the semantic auto-router spend assertion by @yuneng-berri in #39804
• fix(guardrails): record guardrail information for undecorated custom apply_guardrail overrides by @devin-ai-integration[bot] in #39727
• feat(responses): honor supported_endpoints /v1/responses opt-in for OpenAI-compatible deployments by @devin-ai-integration[bot] in #39725
• fix(router): hold max_parallel_requests slot until streaming response is exhausted or closed by @devin-ai-integration[bot] in #39859
• fix(proxy): retry deadlocks and requeue spend logs on any DB write error by @devin-ai-integration[bot] in #39883
• fix(mcp): reject URL credentials for none auth by @tin-berri in #39926
• fix(hide-secrets): stop redacting benign identifiers by @yucheng-berri in #39879
• perf(logging): scan large base64 payloads for log truncation off the event loop by @devin-ai-integration[bot] in #39890
• fix(proxy): make the invalid-model 403 path cheap under a burst of rejections by @devin-ai-integration[bot] in #39892
• test(e2e): cover Anthropic /chat/completions streaming and tool calls by @yuneng-berri in #39916
• fix(router): coordinate async and sync failure handlers at remaining router call sites by @devin-ai-integration[bot] in #39887
• fix(cloudzero): infer daily batch schema from every row by @yucheng-berri in #39871
• fix(cloudzero): preserve late resource tags by @yucheng-berri in #39873
• feat(ui): deep link guardrail detail with ?guardrail= on guardrails pages by @ryan-crabbe-berri in #39930
• test: repair two CI tests broken by intentional changes by @yuneng-berri in #39932
• feat(terraform/gcp): dependencies-only mode and bring-your-own-network for GKE by @devin-ai-integration[bot] in #39695
• test(e2e): cover Anthropic and OpenAI prompt caching, Cohere embeddings, and costed /openai chat passthrough by @yuneng-berri in #39920
• test(e2e): cover key spend reset, regenerate grace period, and the llm_api_routes grant by @yuneng-berri in #39917
• test(e2e/ui): select 50 rows per page before asserting the Tags and Model Hub tables overflow by @yuneng-berri in #39934
• feat(auto-router): decouple compression between the routing decision and the model call by @moe-berri in #39823
• feat(mcp): renew the stored SSO identity assertion behind ID-JAG by @yassin-berriai in #35401
• feat(mcp): warn when an oauth2_id_jag server outruns the SSO provider's assertion capture by @yassin-berriai in #35394
• perf: lazy-load SDK symbols so import litellm stays under 60 MB RSS by @devin-ai-integration[bot] in #39121
• test(e2e): cover presidio post_call, tool_permission, and weave logging cells by @yucheng-berri in #39279
• fix(ci): grant pull_requests write for release wheel reporter by @cursor[bot] in #39922
• feat(guardrails): add non-blocking flag() verdict to custom code guardrails by @devin-ai-integration[bot] in #39728
• feat(proxy): serve Prometheus /metrics from a separate process via --prometheus_metrics_port by @devin-ai-integration[bot] in #39889
• fix(mcp): scan and mask MCP tool call arguments in unified guardrails by @yassin-berriai in #35142
• fix(proxy): reject ambiguous name or alias keys in mcp_tool_permissions on write by @devin-ai-integration[bot] in #39947
• test(e2e): stream a longer /v1/messages reply so the delta-count pin has margin by @yuneng-berri in #39946
• feat(ui): show guardrail usage units and cost on the Guardrails Monitor by @ryan-crabbe-berri in #39853
• fix(ui): show indirectly granted and name-keyed MCP servers in the tool matrix by @yassin-berriai in #35154
• test(e2e): prove Vertex context caching on the first cold call and on the spend row by @yuneng-berri in #39938
• feat(ocr): add Cohere Parse support for cohere and azure_ai by @mateo-berri in #39862
• fix(proxy): keep guardrail cost in spend on cache hits by @ryan-crabbe-berri in #39960
• test(e2e): judge /v1/messages streaming on the clock, not on the provider's delta count by @yuneng-berri in #39953
• fix(ui): remove unreachable AI Hub dialog that put the session key in a URL by @ryan-crabbe-berri in #39968
• fix(ui): read Usage Total Requests tile from gateway request counts by @devin-ai-integration[bot] in #39963
• revert: perf: lazy-load SDK symbols so import litellm stays under 60 MB RSS (#39121) by @yuneng-berri in #39969
• chore: rebuild Admin UI bundle for the next release by @yuneng-berri in #39959
• chore(ci): promote internal staging to main by @yuneng-berri in #39849
• fix(docker): ship pymongo in the proxy images for the MongoDB vector store by @yuneng-berri in #39994

New Contributors

• @amasen02 made their first contribution in #39729

Full Changelog: v1.101.0-dev.2...v1.101.0-rc.1