v1.102.0-dev.1

Verify Docker Image Signature

All LiteLLM Docker images are signed with cosign. Every release is signed with the same key introduced in commit 0112e53.

Verify using the pinned commit hash (recommended):

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
  ghcr.io/berriai/litellm:v1.102.0-dev.1

Verify using the release tag (convenience):

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/v1.102.0-dev.1/cosign.pub \
  ghcr.io/berriai/litellm:v1.102.0-dev.1

Expected output:

The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key

---

What's Changed

• feat(policy_engine): execute post_call guardrail pipelines on streaming responses by @mateo-berri in #38788
• fix(policy_engine): apply post_call pipeline text rewrites on streams by @mateo-berri in #39233
• fix(proxy): redact provider keys from pass-through failure tracebacks by @mateo-berri in #39964
• fix(file_search): scope emulated file_search to the request's vector stores by @mateo-berri in #39972
• feat(cli): add lite pi to run the pi coding agent through the proxy by @ryan-crabbe-berri in #36841
• feat(pricing): add GovCloud pricing for every live but unpriced Bedrock model by @mateo-berri in #39764
• fix(azure_sentinel): split batches under the 1MB ingestion cap by @yucheng-berri in #39880
• feat(guardrails): add inspect_embeddings toggle for AIM and Cato by @yucheng-berri in #39918
• refactor(ui): route the sidebar by pathname and shrink the ?page= shim to a redirect table by @ryan-crabbe-berri in #39978
• fix(anthropic): keep provider_specific_fields off the native /v1/messages wire by @tin-berri in #39967
• fix(headroom): inject headroom_retrieve only for service-declared ccr_hashes and keep assistant content blocks intact by @tin-berri in #39974
• fix(auto_router): build the semantic route layer off the event loop by @moe-berri in #39954
• fix(adaptive_router): fall back to model_info for cost-weighted scoring by @moe-berri in #39957
• fix(adaptive_router): add the persisted delta to the cold-start prior on load by @moe-berri in #39955
• feat(mcp): add opt-in per-server oauth relay discovery by @tin-berri in #39936
• refactor(ui): render the Virtual Keys page without the legacy user dashboard by @ryan-crabbe-berri in #39991
• feat(router): gate heuristic v1 tuning by @tin-berri in #39952
• fix(docker): ship pymongo in the proxy images for the MongoDB vector store by @yuneng-berri in #39995
• fix(router): treat a routing entry with no latency samples as zero latency by @mateo-berri in #39970
• ci(e2e): run a PR's changed e2e tests three times behind a human-approved environment by @yuneng-berri in #39416
• feat(vertex): add Lyria model support by @emerzon in #30856
• ci: run unit tests on Python 3.12 by @yujonglee-berri in #39989
• fix(anthropic_responses): preserve Responses refusal blocks in Anthropic messages translation by @Atharva-Kanherkar in #39723
• fix(batches): account a batch's cost once, from the first retrieve that sees it final by @mateo-berri in #39980
• feat(cost-map): add azure_ai/gpt-6-astra Foundry pricing by @mateo-berri in #39983
• chore(lint): stop ratcheting *-budget.json on PR branches by @devin-ai-integration[bot] in #39937
• ci: simplify Rust checks and remove wheel PR comments by @yujonglee-berri in #39975
• fix(router): skip the refusing deployment when retrying a non-transient error by @mateo-berri in #40014
• feat(mistral): add text-to-speech support for /v1/audio/speech by @mateo-berri in #38755
• fix(mcp): apply key and team guardrails to MCP tool calls by @mubashir1osmani in #39629
• test(ui): pin wire contracts for key, model and MCP server forms by @yuneng-berri in #40019
• refactor(rust): remove per-request enablement arguments by @yujonglee-berri in #39928
• test(ocr): trace callback lifecycle parity by @yujonglee-berri in #40063
• fix(router): serialize heuristic tuning quota filters for Prisma by @yuneng-berri in #40140
• docs(claude): have runs embed their own QA screenshots on visual changes by @mateo-berri in #39800
• fix(oci): stream Cohere tool-calling answers once by @mateo-berri in #39965
• fix(async): move remote image fetches off the event loop for Snowflake, Bedrock invoke Claude, Mantle and Gemini by @mateo-berri in #39839
• fix(ai-gateway): dial upstream WebSockets over an explicit rustls provider by @mateo-berri in #39530
• fix(cost): bill realtime reasoning tokens nested in text_tokens once by @mateo-berri in #39850
• fix: keep provider id and metadata on Responses API bridged chat completions by @mateo-berri in #39981
• fix(make check): lint the test tree on tests-only changes like CI does by @mateo-berri in #39870
• fix(ci): stop the auto-close duplicates job colliding with the required test check by @mateo-berri in #40015
• fix: capture provider request id in failure logging payloads by @yucheng-berri in #40045
• test(ui): make navigation smoke resilient to router refactors by @yuneng-berri in #40026
• test(e2e/ui): cover member role and budget edits, member permission delegation, and team guardrail removal by @yuneng-berri in #40042
• test(ocr): add SDK callback E2E parity by @yujonglee-berri in #40061
• test(e2e): prove the virtual key lifecycle on every gateway replica by @yuneng-berri in #40023
• fix(spend_logs): keep partition DDL transactions alive for their statement timeout by @devin-ai-integration[bot] in #40098
• feat(proxy): resolve root_path per request from a configured prefix list (SERVER_ROOT_PATHS) by @gym-cmd in #35935
• fix(router): preserve heuristic defaults across releases by @tin-berri in #40007
• fix(proxy): enforce key and team model access on Bedrock passthrough routes (internal copy of #34244) by @mateo-berri in #39660
• feat(ci): add the cost map guard check by @mateo-berri in #39835
• feat(ui): add key-scoped auto-router usage tab by @tin-berri in #39999
• fix(ui): hide the Create Vector Store flow from non proxy admins by @devin-ai-integration[bot] in #40148
• fix(ui): send empty vector_stores when the last team vector store is removed by @devin-ai-integration[bot] in #40144
• feat(skills): semantic search over the LiteLLM-hosted skill registry by @yassin-berriai in #39401
• fix(ui): show inherited MCP servers on the internal user editor and flag access groups with no members by @devin-ai-integration[bot] in #40036
• feat(keys): allow editing soft budget on existing keys by @devin-ai-integration[bot] in #39002
• fix(guardrails): block Prompt Security file modifications by @davida-ps in #38204
• fix(router): keep provider response headers on streaming chat completions by @yucheng-berri in #40091
• ci: prepare workflows for main default branch by @yuneng-berri in #40149
• fix(proxy): let authorized internal users open vector store details by @devin-ai-integration[bot] in #40150
• feat(ui): make automatic auto-router setup discoverable and show what it configured by @tin-berri in #40146
• fix(docs): fix stale file paths in ARCHITECTURE.md by @jrlprost in #40157
• fix(guardrails): accept on_violation block and alert for mcp_security by @devin-ai-integration[bot] in #40155
• fix(spend): price caching savings on the billed request basis by @tin-berri in #40160
• fix(responses): drop unsupported reasoning param for openai non-reasoning models by @mateo-berri in #38842
• fix(guardrails): resolve generateContent routes and async-first passthrough call types by @mateo-berri in #38869
• fix(mcp): forward staged credentials on /mcp-rest/test/connection like /test/tools/list by @mateo-berri in #38806
• fix(proxy): register SkillsInjectionHook at proxy startup instead of import time by @mateo-berri in #38914
• feat(ui): list the ChatGPT subscription provider in the Add Model form by @ryan-crabbe-berri in #40170
• test(guardrails): pin the presidio spend-log record and the UI's masked-entity persistence by @yuneng-berri in #40038
• test(e2e/ui): automate the RC checklist's Presidio guardrail walk by @yuneng-berri in #40041
• ci: follow the default branch in development tooling by @yuneng-berri in #40162
• fix(mcp): encrypt stored static headers and stdio environment by @tin-berri in #40164
• feat(fireworks_ai): add native Responses API config by @mateo-berri in #39826
• feat(ui): itemize auto-router classification spend by @tin-berri in #40168
• fix(bedrock): keep x-amzn-RequestId on chat error responses by @yucheng-berri in #40089
• fix(mcp): clear error when an agent-bound key is denied a scoped MCP server + agent MCP grants in the UI by @mateo-berri in #39234
• fix(anthropic): skip one-shot Claude Code cache injection by @tin-berri in #40175
• feat(complexity_router): add declarative custom dimensions to the heuristic scorer by @tin-berri in #40156
• fix(proxy): log budget reservation notice once at config load by @yucheng-berri in #40167
• test: add Rust extension pytest contract by @yujonglee-berri in #40181
• test: drop tests that only pin static cost-map values by @devin-ai-integration[bot] in #40195
• fix(vertex_ai): support fine-tuned Gemini endpoints in managed batches by @mubashir1osmani in #39668
• fix(spend): compare auto-router targets by deployment identity by @tin-berri in #40206
• feat(ui): show auto-router classification rate by @tin-berri in #40192
• feat(mcp): start the named server's OAuth directly for a resource-scoped gateway flow by @tin-berri in #39933
• fix(router): rank streaming latency routing by raw TTFT, not TTFT per token by @clement-paradex in #40202
• fix(tool_permission): log expected skip and deny events below WARNING by @devin-ai-integration[bot] in #40208
• fix(proxy): kill the whole prisma process group when a boot migration command times out by @mateo-berri in #39509
• fix(proxy): load db credentials in the model reconcile so a worker never serves a model before its credential by @mateo-berri in #39876
• fix(router): give cooldowns their own cache so siblings see a bench in ~1s by @mateo-berri in #40025
• fix(least-busy): share in-flight request counts across proxy workers by @mateo-berri in #40009
• fix(proxy-extras): rebuild indexes left INVALID by a migration deadlock by @mateo-berri in #39384
• fix(guardrails): keep guardrail telemetry when a policy pipeline blocks or modifies the response by @devin-ai-integration[bot] in #40211
• fix(ocr): run post-call logging hooks by @yujonglee-berri in #40154
• fix(guardrails): allow framework-supported logging-only mode by @yujonglee-berri in #40267
• fix(responses bridge): keep mid-conversation system messages in input instead of folding them into instructions by @devin-ai-integration[bot] in #40269
• fix(proxy): initialize string success/failure callbacks at startup after config load by @devin-ai-integration[bot] in #38226
• fix(responses): record spend for native Responses API WebSocket sessions by @devin-ai-integration[bot] in #38856
• feat(complexity_router): rebalance heuristic weights in the dashboard and grade custom dimensions by match count by @tin-berri in #40205
• feat(deploy): metrics sidecar and separate metrics port in Helm and Terraform by @devin-ai-integration[bot] in #40163
• feat(router): resolve max_tokens to the tier model's ceiling on auto-routed requests by @tin-berri in #40209
• chore: bump litellm-proxy-extras 0.4.94 -> 0.4.95, litellm 1.101.0 -> 1.102.0 by @yuneng-berri in #40287
• feat(otel): make the OTel v2 trace export URL configurable by @devin-ai-integration[bot] in #40286
• fix(router): keep per-request routing_strategy override selectors out of global callbacks by @devin-ai-integration[bot] in #40229
• fix(router): count allowed_fails in the shared router cache so multi-worker proxies bench a deployment fleet-wide by @devin-ai-integration[bot] in #40224
• fix(router): let simple-shuffle weight by any deployment's weight/rpm/tpm by @devin-ai-integration[bot] in #40222
• fix(router): give cost-based routing its own cache key so it stops overwriting latency samples by @devin-ai-integration[bot] in #40225
• fix(router): keep deployment tags out of retry and fallback tag routing by @devin-ai-integration[bot] in #40226
• fix(proxy): keep team member budget enforced at the cap and across Redis counter expiry by @devin-ai-integration[bot] in #40304
• fix(utils): honor string drop_params values from config and DB deployments by @mateo-berri in #33738
• fix(policy_engine): execute post_call guardrail pipelines on responses and streams by @mateo-berri in #38721
• feat(ui): batch observability on the logs page by @mubashir1osmani in #39626
• feat(otel): add http/json export protocol for OTel v2 traces by @devin-ai-integration[bot] in #40290
• feat: move MongoDB vector search to an optional sidecar (BETA) by @yuneng-berri in #40203
• perf: move Anthropic, Vertex Anthropic, Ollama and HF template fetches off the event loop by @devin-ai-integration[bot] in #40311
• fix(router): move retry-policy retries off the refusing deployment on every router entrypoint by @devin-ai-integration[bot] in #40306
• fix(proxy): default max_idle_connection_lifetime on componentized DB URLs by @devin-ai-integration[bot] in #40285
• fix(a2a): forward caller identity headers on message/send and message/stream by @yassin-berriai in #40305
• feat(team): let a team admin manage their own team's logging callbacks by @yucheng-berri in #37667
• fix(proxy): give each spend-log queue monitor its own flush event by @mateo-berri in #39556
• fix(mlflow): prevent _stream_id_to_span leak and mlflow 2.x end_trace TypeError by @devin-ai-integration[bot] in #39049
• feat(otel v2): send a key's or team's whole trace to its own destination by @yucheng-berri in #39654
• chore(ui): update dashboard dependencies by @yuneng-berri in #40312
• feat(bedrock): add TwelveLabs Marengo Embed 3.0 embeddings by @devin-ai-integration[bot] in #40180
• fix(proxy): stop shipping the literal string "None" as error type and param by @mateo-berri in #39536
• chore(ci): merge main into internal staging by @yuneng-berri in #40323
• test(azure_sentinel): pin batch_size as a per-request bound under concurrent events by @devin-ai-integration[bot] in #40320
• fix(fireworks_ai): fold instructions and developer items into one leading system message on the Responses path by @devin-ai-integration[bot] in #40268
• feat(auto_router): opt-in NON_REASONING tier below SIMPLE by @moe-berri in #40273
• fix(budget_reservation): don't reserve budget on token counting routes by @devin-ai-integration[bot] in #36718
• feat(mcp): add schema discovery proxy mode by @tin-berri in #40298
• test(proxy): fix Prisma timeout cleanup after subreaper tests by @yuneng-berri in #40333
• fix(otel v2): restore the Datadog auth span and the last-wins callback merge by @devin-ai-integration[bot] in #40335
• chore(ci): extend diskcache scan exception to October 1 by @yuneng-berri in #40336
• fix(anthropic): key the /v1/messages prompt cache on Claude Code's session_id only by @ryan-crabbe-berri in #40342
• feat(auto-router): refresh family reasoning presets by @tin-berri in #40341
• feat(cost_map): report which revision of the price map the proxy is serving by @devin-ai-integration[bot] in #40179
• test(mcp): fix proxy fixture isolation after manager reload by @yuneng-berri in #40347
• chore: bump litellm-enterprise 0.1.65 -> 0.1.66 by @yuneng-berri in #40334
• chore(ci): promote internal staging to main by @yuneng-berri in #40307

New Contributors

• @Atharva-Kanherkar made their first contribution in #39723
• @gym-cmd made their first contribution in #35935
• @jrlprost made their first contribution in #40157
• @clement-paradex made their first contribution in #40202

Full Changelog: v1.101.0-rc.1...v1.102.0-dev.1