Releases: BerriAI/litellm
Releases · BerriAI/litellm
v1.72.6-stable
What's Changed
- Fix default team settings by @NANDINI-star in #11888
Full Changelog: v1.72.6.post1-nightly...v1.72.6-stable
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.72.6-stable
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 250.0 | 269.27781931947453 | 6.111834388077504 | 0.0 | 1828 | 0 | 215.86210600003142 | 1630.9297619999938 |
| Aggregated | Passed ✅ | 250.0 | 269.27781931947453 | 6.111834388077504 | 0.0 | 1828 | 0 | 215.86210600003142 | 1630.9297619999938 |
Contributors
NANDINI-star
Assets 4
v1.72.6.post1-nightly
Full Changelog: v1.72.6.dev1...v1.72.6.post1-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.72.6.post1-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 190.0 | 209.8013988365269 | 6.275681933110413 | 0.0 | 1878 | 0 | 167.48262099997646 | 1487.4784890000115 |
| Aggregated | Passed ✅ | 190.0 | 209.8013988365269 | 6.275681933110413 | 0.0 | 1878 | 0 | 167.48262099997646 | 1487.4784890000115 |
v1.72.6.devSCIM
What's Changed
- feat(azure): Make Azure AD scope configurable by @kjoth in #11621
- Litellm stable docs 06 14 2025 p2 by @krrishdholakia in #11738
- Release note updates + Responses API Bridge improvements by @krrishdholakia in #11740
- VertexAI Anthropic - streaming passthrough cost tracking by @krrishdholakia in #11734
- Fix PrometheusLogger label_filters initialization for non-premium users by @colesmcintosh in #11764
- Add Vertex Imagen-4 models by @emerzon in #11767
- Users page buttons repositioned by @NANDINI-star in #11771
- #11748: Added Mistral Small to BEDROCK_CONVERSE_MODELS for Converse A… by @shagunb-acn in #11760
- [Security] Fixes for docs by @ishaan-jaff in #11776
- [Security] - Add Trivy Security Scan for UI + Docs folder - remove all vulnerabilities by @ishaan-jaff in #11778
- [Fix] Pass through - Langfuse don't log request to Langfuse passthrough on Langfuse by @ishaan-jaff in #11768
- [Deps] Fix aiohttp version requirement by @ishaan-jaff in #11777
- AWS credentials no longer mandatory by @MadsRC in #11765
- build(deps): bump next from 14.2.26 to 14.2.30 in /ui/litellm-dashboard by @dependabot in #11720
- feat: update the feature of ollama_embeddings to work on a sync api by @Abiji-2020 in #11746
- [Feat] Day-0 Support for OpenAI Re-usable prompts Responses API by @ishaan-jaff in #11782
- SSO - Allow passing additional headers + Spend Tags - automatically track spend by user agent (allows cost tracking for claude code) by @krrishdholakia in #11781
- JWT Auth - correctly return user email + UI Model Update - Allow editing model access group for existing model by @krrishdholakia in #11783
- Allow
/modelsto return correct models for custom wildcard prefixes by @krrishdholakia in #11784 - Fix JSX syntax error in documentation causing Vercel deployment failure by @colesmcintosh in #11818
- [Fix] Bug Fix for using prom metrics config by @ishaan-jaff in #11779
- [Bug Fixes] MCP - using MCPs defined on config.yaml + fix for MCP error Team doesn't exist in cache by @ishaan-jaff in #11824
- new gemini model pricing + a few openrouter models model_prices_and_context_window.json by @salzubi401 in #11803
- Update bedrock guardrail docs by @orolega in #11826
- [Feat] v2 Pass through endpoints - Add support for subroutes for pass through endpoints + Cleaned up UI by @ishaan-jaff in #11827
- Fix vertex ai claude thinking params by @X4tar in #11796
- Implement health check backend API and storage functionality - fix ci/cd by @colesmcintosh in #11852
- [Fix] v1/messages endpoint always uses us-central1 with vertex_ai-anthropic models by @ishaan-jaff in #11831
- Fix #11856: Update billing.md docs to call the new GPT-4o model by @karen-veigas in #11858
- Add LiteLLM_HealthCheckTable to database schema by @colesmcintosh in #11677
- [SCIM] Add Error handling for existing user on SCIM by @ishaan-jaff in #11862
New Contributors
- @kjoth made their first contribution in #11621
- @shagunb-acn made their first contribution in #11760
- @MadsRC made their first contribution in #11765
- @Abiji-2020 made their first contribution in #11746
- @salzubi401 made their first contribution in #11803
- @orolega made their first contribution in #11826
- @X4tar made their first contribution in #11796
- @karen-veigas made their first contribution in #11858
Full Changelog: v1.72.6.dev1...v1.72.6.devSCIM
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.72.6.devSCIM
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 190.0 | 215.1720637640139 | 6.262237464870193 | 0.0 | 1873 | 0 | 171.28891599998042 | 1800.7898239999918 |
| Aggregated | Passed ✅ | 190.0 | 215.1720637640139 | 6.262237464870193 | 0.0 | 1873 | 0 | 171.28891599998042 | 1800.7898239999918 |
Contributors
MadsRC, krrishdholakia, and 12 other contributors
Assets 4
v1.72.6.SCIM2
Full Changelog: v1.72.6.devSCIM...v1.72.6.SCIM2
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.72.6.SCIM2
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 190.0 | 213.45712869978374 | 6.190773809263607 | 0.0 | 1852 | 0 | 171.36217200004467 | 1296.009626 |
| Aggregated | Passed ✅ | 190.0 | 213.45712869978374 | 6.190773809263607 | 0.0 | 1852 | 0 | 171.36217200004467 | 1296.009626 |
v1.72.6.rc
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.72.6.rc
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 190.0 | 206.1983524503451 | 6.292694671664114 | 0.0 | 1883 | 0 | 168.78388700001778 | 1487.5943659999962 |
| Aggregated | Passed ✅ | 190.0 | 206.1983524503451 | 6.292694671664114 | 0.0 | 1883 | 0 | 168.78388700001778 | 1487.5943659999962 |
What's Changed
- [Docs] v1.72.2.rc by @ishaan-jaff in #11519
- Support env var vertex credentials for passthrough + ignore space id on watsonx deployment (throws Json validation errors) by @krrishdholakia in #11527
- Ensure consistent 'created' across all chunks + set tool call id for ollama streaming calls by @krrishdholakia in #11528
- Update enduser spend and budget reset date based on budget duration by @laurien16 in #8460
- feat: add .cursor to .gitignore by @colesmcintosh in #11538
- Add gpt-4o-audio-preview-2025-06-03 pricing configuration by @colesmcintosh in #11560
- [Docs] Fix incorrect reference to database_url as master_key by @fengbohello in #11547
- Update documentation for configuring web search options in config.yaml by @colesmcintosh in #11537
- [Bug fix]: aiohttp fixes for transfer encoding error on aiohttp transport by @ishaan-jaff in #11561
- [Feat] Add
reasoning_effortsupport for perplexity models by @ishaan-jaff in #11562 - Make all commands show server URL by @msabramo in #10801
- Simplify
management_cli.mdCLI docs by @msabramo in #10799 - Fix: Adds support for choosing the default region based on where the model is available by @ishaan-jaff in #11566
- [Feat] Add Lasso Guardrail to LiteLLM by @ishaan-jaff in #11565
- Fix gemini tool call indexes by @lowjiansheng in #11558
- Show remaining users on UI + prevent early stream stopping for gemini requests by @krrishdholakia in #11568
- Add VertexAI
claude-opus-4+ Assign users to orgs on creation by @krrishdholakia in #11572 - Pangea/kl/udpate readme by @lapinek in #11570
- Update README.md so docker compose will work as described by @yanwork in #11586
- Add support for new Mistral Magistral models (magistral-medium-2506 and magistral-small-2506) by @colesmcintosh in #11588
- (fix:exception_mapping_utils.py) fix sglang rate limit error issue by @dhs-shine in #11575
- [Feat] LiteLLM Allow setting Uvicorn Keep Alive Timeout by @ishaan-jaff in #11594
- [Bug Fix] No module named 'diskcache' by @ishaan-jaff in #11600
- [Feat] UI - Add controls for MCP Permission Management by @ishaan-jaff in #11598
- [Feat] New LLM API Endpoint - Add List input items for Responses API by @ishaan-jaff in #11602
- Add new o3 models pricing by @krrishdholakia in #11606
- [UI] Polish New MCP Server Add Form by @ishaan-jaff in #11604
- Litellm dev 06 10 2025 p2 by @krrishdholakia in #11605
- Add VertexAI Anthropic passthrough - cost calculation, token tracking by @krrishdholakia in #11611
- fix(internal_user_endpoints.py): support user with
+in email on user info + handle empty string for arguments on gemini function calls by @krrishdholakia in #11601 - Fix: passes api_base, api_key, litellm_params_dict to custom_llm embedding methods by @ElefHead in #11450
- Add Admin-Initiated Password Reset Flow by @NANDINI-star in #11618
- fix inference endpoints (#11630) by @ishaan-jaff in #11631
- [UI] Add Deepgram provider to supported providers list and mappings by @ishaan-jaff in #11634
- [Bug Fix] Add audio/ogg mapping for Audio MIME types by @ishaan-jaff in #11635
- [Feat] Add Background mode for Responses API - OpenAI, AzureOpenAI by @ishaan-jaff in #11640
- [Feat] Add provider specific params for
deepgram/by @ishaan-jaff in #11638 - [Feat] MCP - Add support for
streamablehttp_clientMCP Servers by @ishaan-jaff in #11628 - [Feat] Perf fix - ensure deepgram provider uses async httpx calls by @ishaan-jaff in #11641
- Trim the long user ids on the the keys page by @NANDINI-star in #11488
- Enable System Proxy Support for aiohttp Transport by @idootop in #11616
- GA Multi-instance rate limiting v2 Requirements + New - specify token rate limit type - output / input / total by @krrishdholakia in #11646
- Add bridge for /chat/completion -> /responses API by @krrishdholakia in #11632
- Convert scientific notation str to int + Bubble up azure content filter results by @krrishdholakia in #11655
- feat(helm): [#11648] support extraContainers in migrations-job.yaml by @stevenaldinger in #11649
- Correct success message when user creates new budget by @vuanhtu52 in #11608
- fix: Do not add default model on tag based-routing when valid tag by @thiagosalvatore in #11454
- Fix default user settings by @NANDINI-star in #11674
- [Pricing] add azure/gpt-4o-mini-transcribe models by @ishaan-jaff in #11676
- Enhance Mistral model support with reasoning capabilities by @colesmcintosh in #11642
- [Feat] MCP expose streamable https endpoint for LiteLLM Proxy by @ishaan-jaff in #11645
- change space_key header to space_id for Arize by @vanities in #11595
- Add performance indexes to LiteLLM_SpendLogs for analytics queries by @colesmcintosh in #11675
- Revert "Add performance indexes to LiteLLM_SpendLogs for analytics queries" by @krrishdholakia in #11683
- [Feat] Use dedicated Rest endpoints for list, calling MCP tools by @ishaan-jaff in #11684
- Chat Completions <-> Responses API Bridge Improvements by @krrishdholakia in #11685
- [UI] Fix MCP Server Table to Match Existing Table Pattern by @ishaan-jaff in #11691
- Logging: prevent double logging logs when bridge is used (anthropic <-> chat completion OR chat completion <-> responses api) by @krrishdholakia in #11687
- fix(vertex_ai): support global location in vertex ai passthrough by @alvarosevilla95 in #11661
- [Feat] UI Allow editing mcp servers by @ishaan-jaff in #11693
- [Feat] UI - Allow setting MCP servers when creating keys, teams by @ishaan-jaff in #11711
- [Feat] Add Authentication + Permission Management for MCP List, Call Tool Ops by @ishaan-jaff in #11682
- Add Live Tail Feature to Logs View by @NANDINI-star in #11712
- [Feat] Add Connect to MCP Page by @ishaan-jaff in #11716
- Enterprise feature preview improvement on Audit Logs by @NANDINI-star in #11715
- Align Model Connection Success Icon and Text by @NANDINI-star in #11717
- fix(prometheus.py): fix total requests increment + add semantic tests fo… by @krrishdholakia in #11718
- Anthropic - add 'prefix' to start of assistant content + Add model access groups on UI by @krrishdholakia in #11719
- Add anthropic 'none' tool choice param support by @krrishdholakia in #11695
- [Feat] UI - Add back favicon by @ishaan-jaff in #11728
- Time taken column logs by @gbrian in #11723
- UI - Fix remaining users activit...
Contributors
msabramo, gbrian, and 19 other contributors
Assets 4
v1.72.6.dev1
What's Changed
- Time taken column logs by @gbrian in #11723
- UI - Fix remaining users activity if no limit + allow filtering by model access groups by @krrishdholakia in #11730
- [Feat] Prometheus - Allow controlling exposed metrics and labels by @ishaan-jaff in #11489
- Add Langfuse OpenTelemetry Integration by @colesmcintosh in #11607
- Fixed grok-3-mini to not use stop tokens by @Naggafin in #11563
- Implement health check backend API and storage functionality by @colesmcintosh in #11678
- fix: re-export get_valid_models utils in init file by @jsondai in #11721
- [Bug Fix] Ensure /redis/info works on GCP Redis by @ishaan-jaff in #11732
- [Fix] JWT - Fix error when team member already part of team by @ishaan-jaff in #11735
- [Bug Fix] Rollback dd-trace version by @ishaan-jaff in #11736
- Prometheus - fix request increment + add route tracking for streaming requests by @krrishdholakia in #11731
- Litellm stable release 06 14 2025 by @krrishdholakia in #11737
New Contributors
- @gbrian made their first contribution in #11723
- @Naggafin made their first contribution in #11563
- @jsondai made their first contribution in #11721
Full Changelog: v1.72.6-nightly...v1.72.6.dev1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.72.6.dev1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 240.0 | 263.48009916102654 | 6.12478893553391 | 0.0 | 1832 | 0 | 214.1671110000516 | 1761.2241329999847 |
| Aggregated | Passed ✅ | 240.0 | 263.48009916102654 | 6.12478893553391 | 0.0 | 1832 | 0 | 214.1671110000516 | 1761.2241329999847 |
Contributors
gbrian, Naggafin, and 4 other contributors
Assets 4
v1.72.6-nightly
What's Changed
- [Feat] UI - Add back favicon by @ishaan-jaff in #11728
Full Changelog: 1.72.6.rc-draft...v1.72.6-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.72.6-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 250.0 | 265.04106068733097 | 6.154497639770629 | 0.0 | 1839 | 0 | 214.5358220000162 | 1687.9459110000425 |
| Aggregated | Passed ✅ | 250.0 | 265.04106068733097 | 6.154497639770629 | 0.0 | 1839 | 0 | 214.5358220000162 | 1687.9459110000425 |
Contributors
ishaan-jaff
Assets 4
[DRAFT] 1.72.6.rc
What's Changed
- [Docs] v1.72.2.rc by @ishaan-jaff in #11519
- Support env var vertex credentials for passthrough + ignore space id on watsonx deployment (throws Json validation errors) by @krrishdholakia in #11527
- Ensure consistent 'created' across all chunks + set tool call id for ollama streaming calls by @krrishdholakia in #11528
- Update enduser spend and budget reset date based on budget duration by @laurien16 in #8460
- feat: add .cursor to .gitignore by @colesmcintosh in #11538
- Add gpt-4o-audio-preview-2025-06-03 pricing configuration by @colesmcintosh in #11560
- [Docs] Fix incorrect reference to database_url as master_key by @fengbohello in #11547
- Update documentation for configuring web search options in config.yaml by @colesmcintosh in #11537
- [Bug fix]: aiohttp fixes for transfer encoding error on aiohttp transport by @ishaan-jaff in #11561
- [Feat] Add
reasoning_effortsupport for perplexity models by @ishaan-jaff in #11562 - Make all commands show server URL by @msabramo in #10801
- Simplify
management_cli.mdCLI docs by @msabramo in #10799 - Fix: Adds support for choosing the default region based on where the model is available by @ishaan-jaff in #11566
- [Feat] Add Lasso Guardrail to LiteLLM by @ishaan-jaff in #11565
- Fix gemini tool call indexes by @lowjiansheng in #11558
- Show remaining users on UI + prevent early stream stopping for gemini requests by @krrishdholakia in #11568
- Add VertexAI
claude-opus-4+ Assign users to orgs on creation by @krrishdholakia in #11572 - Pangea/kl/udpate readme by @lapinek in #11570
- Update README.md so docker compose will work as described by @yanwork in #11586
- Add support for new Mistral Magistral models (magistral-medium-2506 and magistral-small-2506) by @colesmcintosh in #11588
- (fix:exception_mapping_utils.py) fix sglang rate limit error issue by @dhs-shine in #11575
- [Feat] LiteLLM Allow setting Uvicorn Keep Alive Timeout by @ishaan-jaff in #11594
- [Bug Fix] No module named 'diskcache' by @ishaan-jaff in #11600
- [Feat] UI - Add controls for MCP Permission Management by @ishaan-jaff in #11598
- [Feat] New LLM API Endpoint - Add List input items for Responses API by @ishaan-jaff in #11602
- Add new o3 models pricing by @krrishdholakia in #11606
- [UI] Polish New MCP Server Add Form by @ishaan-jaff in #11604
- Litellm dev 06 10 2025 p2 by @krrishdholakia in #11605
- Add VertexAI Anthropic passthrough - cost calculation, token tracking by @krrishdholakia in #11611
- fix(internal_user_endpoints.py): support user with
+in email on user info + handle empty string for arguments on gemini function calls by @krrishdholakia in #11601 - Fix: passes api_base, api_key, litellm_params_dict to custom_llm embedding methods by @ElefHead in #11450
- Add Admin-Initiated Password Reset Flow by @NANDINI-star in #11618
- fix inference endpoints (#11630) by @ishaan-jaff in #11631
- [UI] Add Deepgram provider to supported providers list and mappings by @ishaan-jaff in #11634
- [Bug Fix] Add audio/ogg mapping for Audio MIME types by @ishaan-jaff in #11635
- [Feat] Add Background mode for Responses API - OpenAI, AzureOpenAI by @ishaan-jaff in #11640
- [Feat] Add provider specific params for
deepgram/by @ishaan-jaff in #11638 - [Feat] MCP - Add support for
streamablehttp_clientMCP Servers by @ishaan-jaff in #11628 - [Feat] Perf fix - ensure deepgram provider uses async httpx calls by @ishaan-jaff in #11641
- Trim the long user ids on the the keys page by @NANDINI-star in #11488
- Enable System Proxy Support for aiohttp Transport by @idootop in #11616
- GA Multi-instance rate limiting v2 Requirements + New - specify token rate limit type - output / input / total by @krrishdholakia in #11646
- Add bridge for /chat/completion -> /responses API by @krrishdholakia in #11632
- Convert scientific notation str to int + Bubble up azure content filter results by @krrishdholakia in #11655
- feat(helm): [#11648] support extraContainers in migrations-job.yaml by @stevenaldinger in #11649
- Correct success message when user creates new budget by @vuanhtu52 in #11608
- fix: Do not add default model on tag based-routing when valid tag by @thiagosalvatore in #11454
- Fix default user settings by @NANDINI-star in #11674
- [Pricing] add azure/gpt-4o-mini-transcribe models by @ishaan-jaff in #11676
- Enhance Mistral model support with reasoning capabilities by @colesmcintosh in #11642
- [Feat] MCP expose streamable https endpoint for LiteLLM Proxy by @ishaan-jaff in #11645
- change space_key header to space_id for Arize by @vanities in #11595
- Add performance indexes to LiteLLM_SpendLogs for analytics queries by @colesmcintosh in #11675
- Revert "Add performance indexes to LiteLLM_SpendLogs for analytics queries" by @krrishdholakia in #11683
- [Feat] Use dedicated Rest endpoints for list, calling MCP tools by @ishaan-jaff in #11684
- Chat Completions <-> Responses API Bridge Improvements by @krrishdholakia in #11685
- [UI] Fix MCP Server Table to Match Existing Table Pattern by @ishaan-jaff in #11691
- Logging: prevent double logging logs when bridge is used (anthropic <-> chat completion OR chat completion <-> responses api) by @krrishdholakia in #11687
- fix(vertex_ai): support global location in vertex ai passthrough by @alvarosevilla95 in #11661
- [Feat] UI Allow editing mcp servers by @ishaan-jaff in #11693
- [Feat] UI - Allow setting MCP servers when creating keys, teams by @ishaan-jaff in #11711
- [Feat] Add Authentication + Permission Management for MCP List, Call Tool Ops by @ishaan-jaff in #11682
- Add Live Tail Feature to Logs View by @NANDINI-star in #11712
- [Feat] Add Connect to MCP Page by @ishaan-jaff in #11716
- Enterprise feature preview improvement on Audit Logs by @NANDINI-star in #11715
- Align Model Connection Success Icon and Text by @NANDINI-star in #11717
- fix(prometheus.py): fix total token increment + add semantic tests fo… by @krrishdholakia in #11718
- Add model access groups on UI by @krrishdholakia in #11719
- Add anthropic 'none' tool choice param support by @krrishdholakia in #11695
New Contributors
- @laurien16 made their first contribution in #8460
- @fengbohello made their first contribution in #11547
- @lapinek made their first contribution in #11570
- @yanwork made their first contribution in #11586
- @dhs-shine made their first contribution in #11575
- @ElefHead made their first contribution in #11450
- @idootop made their first contribution in #11616
- @stevenaldinger made their first contribution in #11649
- @thiagosalvatore made their first contribution in #11454
- @vanities made their first contribution in #11595
- @alvarosevilla95 made their first contribution in #11661
Full Changelog: https://github.com/BerriAI/litellm/compare/v1.72.2-stable......
Contributors
msabramo, yanwork, and 16 other contributors
Assets 2
v1.72.2.devMCP
What's Changed
- fix inference endpoints (#11630) by @ishaan-jaff in #11631
- [UI] Add Deepgram provider to supported providers list and mappings by @ishaan-jaff in #11634
- [Bug Fix] Add audio/ogg mapping for Audio MIME types by @ishaan-jaff in #11635
- [Feat] Add Background mode for Responses API - OpenAI, AzureOpenAI by @ishaan-jaff in #11640
- [Feat] Add provider specific params for
deepgram/by @ishaan-jaff in #11638 - [Feat] MCP - Add support for
streamablehttp_clientMCP Servers by @ishaan-jaff in #11628 - [Feat] Perf fix - ensure deepgram provider uses async httpx calls by @ishaan-jaff in #11641
- Trim the long user ids on the the keys page by @NANDINI-star in #11488
- Enable System Proxy Support for aiohttp Transport by @idootop in #11616
- GA Multi-instance rate limiting v2 Requirements + New - specify token rate limit type - output / input / total by @krrishdholakia in #11646
- Add bridge for /chat/completion -> /responses API by @krrishdholakia in #11632
- Convert scientific notation str to int + Bubble up azure content filter results by @krrishdholakia in #11655
- feat(helm): [#11648] support extraContainers in migrations-job.yaml by @stevenaldinger in #11649
- Correct success message when user creates new budget by @vuanhtu52 in #11608
- fix: Do not add default model on tag based-routing when valid tag by @thiagosalvatore in #11454
- Fix default user settings by @NANDINI-star in #11674
- [Pricing] add azure/gpt-4o-mini-transcribe models by @ishaan-jaff in #11676
- Enhance Mistral model support with reasoning capabilities by @colesmcintosh in #11642
- [Feat] MCP expose streamable https endpoint for LiteLLM Proxy by @ishaan-jaff in #11645
- change space_key header to space_id for Arize by @vanities in #11595
- Add performance indexes to LiteLLM_SpendLogs for analytics queries by @colesmcintosh in #11675
- Revert "Add performance indexes to LiteLLM_SpendLogs for analytics queries" by @krrishdholakia in #11683
- [Feat] Use dedicated Rest endpoints for list, calling MCP tools by @ishaan-jaff in #11684
- Chat Completions <-> Responses API Bridge Improvements by @krrishdholakia in #11685
- [UI] Fix MCP Server Table to Match Existing Table Pattern by @ishaan-jaff in #11691
- Logging: prevent double logging logs when bridge is used (anthropic <-> chat completion OR chat completion <-> responses api) by @krrishdholakia in #11687
- fix(vertex_ai): support global location in vertex ai passthrough by @alvarosevilla95 in #11661
- [Feat] UI Allow editing mcp servers by @ishaan-jaff in #11693
- [Feat] UI - Allow setting MCP servers when creating keys, teams by @ishaan-jaff in #11711
- [Feat] Add Authentication + Permission Management for MCP List, Call Tool Ops by @ishaan-jaff in #11682
- Add Live Tail Feature to Logs View by @NANDINI-star in #11712
- [Feat] Add Connect to MCP Page by @ishaan-jaff in #11716
- Enterprise feature preview improvement on Audit Logs by @NANDINI-star in #11715
New Contributors
- @idootop made their first contribution in #11616
- @stevenaldinger made their first contribution in #11649
- @thiagosalvatore made their first contribution in #11454
- @vanities made their first contribution in #11595
- @alvarosevilla95 made their first contribution in #11661
Full Changelog: v1.72.5.dev1...v1.72.2.devMCP
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.72.2.devMCP
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 220.0 | 241.96992280403583 | 6.294425384311064 | 0.0 | 1883 | 0 | 199.48631400001204 | 1258.8171310000007 |
| Aggregated | Passed ✅ | 220.0 | 241.96992280403583 | 6.294425384311064 | 0.0 | 1883 | 0 | 199.48631400001204 | 1258.8171310000007 |
Contributors
alvarosevilla95, vanities, and 8 other contributors
Assets 4
v1.72.5.dev1
What's Changed
- fix(internal_user_endpoints.py): support user with
+in email on user info + handle empty string for arguments on gemini function calls by @krrishdholakia in #11601 - Fix: passes api_base, api_key, litellm_params_dict to custom_llm embedding methods by @ElefHead in #11450
- Add Admin-Initiated Password Reset Flow by @NANDINI-star in #11618
New Contributors
Full Changelog: v1.72.4-nightly...v1.72.5.dev1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.72.5.dev1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 250.0 | 271.77221555459084 | 6.153062151618842 | 0.0 | 1841 | 0 | 218.69335899998532 | 1399.0517459999978 |
| Aggregated | Passed ✅ | 250.0 | 271.77221555459084 | 6.153062151618842 | 0.0 | 1841 | 0 | 218.69335899998532 | 1399.0517459999978 |
Contributors
ElefHead, krrishdholakia, and NANDINI-star