Releases: BerriAI/litellm
v1.74.9.rc.1
What's Changed
- Litellm release notes 07 27 2025 p1 by @krrishdholakia in #13027
- VertexAI - camelcase optional params for image generation + Anthropic - streaming, always ensure assistant role set on only first chunk by @krrishdholakia in #12889
- Bulk User Edit - additional improvements - edit all users + set 'no-default-models' on all users by @krrishdholakia in #12925
- add X-Initiator header for GitHub Copilot to reduce premium requests by @ckoehler in #13016
- docs - openwebui show how to include reasoning content for gemini models by @ishaan-jaff in #13060
New Contributors
Full Changelog: v1.74.9.rc-draft...v1.74.9.rc.1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.74.9.rc.1
Don't want to maintain your internal proxy? get in touch π
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.74.9.rc.1
Don't want to maintain your internal proxy? get in touch π
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
---|---|---|---|---|---|---|---|---|---|
/chat/completions | Passed β | 130.0 | 168.0418430074381 | 6.293403780045943 | 0.0 | 1882 | 0 | 97.96855799999094 | 1049.3644050000057 |
Aggregated | Passed β | 130.0 | 168.0418430074381 | 6.293403780045943 | 0.0 | 1882 | 0 | 97.96855799999094 | 1049.3644050000057 |
v1.74.9-stable
What's Changed
- Litellm release notes 07 27 2025 p1 by @krrishdholakia in #13027
- VertexAI - camelcase optional params for image generation + Anthropic - streaming, always ensure assistant role set on only first chunk by @krrishdholakia in #12889
- Bulk User Edit - additional improvements - edit all users + set 'no-default-models' on all users by @krrishdholakia in #12925
- add X-Initiator header for GitHub Copilot to reduce premium requests by @ckoehler in #13016
- docs - openwebui show how to include reasoning content for gemini models by @ishaan-jaff in #13060
New Contributors
Full Changelog: v1.74.9.rc-draft...v1.74.9-stable
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.74.9-stable
Don't want to maintain your internal proxy? get in touch π
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
---|---|---|---|---|---|---|---|---|---|
/chat/completions | Passed β | 110.0 | 151.02663461606545 | 6.447232352830848 | 0.0 | 1930 | 0 | 81.53728299998875 | 1408.0881720000207 |
Aggregated | Passed β | 110.0 | 151.02663461606545 | 6.447232352830848 | 0.0 | 1930 | 0 | 81.53728299998875 | 1408.0881720000207 |
litellm_v1.65.4-dev_fix
Full Changelog: v1.65.4-stable...litellm_v1.65.4-dev_fix
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-litellm_v1.65.4-dev_fix
Don't want to maintain your internal proxy? get in touch π
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
---|---|---|---|---|---|---|---|---|---|
/chat/completions | Passed β | 150.0 | 182.3086370464627 | 6.335489722473191 | 0.0 | 1894 | 0 | 112.72804899999755 | 1087.5013490000072 |
Aggregated | Passed β | 150.0 | 182.3086370464627 | 6.335489722473191 | 0.0 | 1894 | 0 | 112.72804899999755 | 1087.5013490000072 |
v1.74.9.rc-draft
What's Changed
- Fix SSO Logout | Create Unified Login Page with SSO and Username/Password Options by @NANDINI-star in #12703
- docs(moonshot): correct base url and document CN-specific endpoint by @magicalne in #12804
- Adding HolmesGPT to projects using LiteLLM by @pavangudiwada in #12798
- feat: Add Lambda AI provider support by @colesmcintosh in #12817
- [UI Bug Fix] Show correct guardrails when editing a team by @ishaan-jaff in #12823
- [Docs] Show correct list of vertex ai mistral models by @ishaan-jaff in #12828
- [LLM Translation - GH Copilot] added dynamic endpoint support by @jugaldb in #12827
- feat: add Morph provider support by @colesmcintosh in #12821
- fix: remove deprecated groq/qwen-qwq-32b and add qwen/qwen3-32b by @aholmberg in #12831
- Add Google Cloud Model Armor guardrail documentation by @colesmcintosh in #12814
- [Feat] Add Recraft Image Generation API Support - New LLM Provider by @ishaan-jaff in #12832
- [Bug Fix] - gemini leaking FD for sync calls with litellm.completion by @ishaan-jaff in #12824
- [Feat] Add fireworks -
fireworks/models/kimi-k2-instruct
by @ishaan-jaff in #12837 - [LLM Translation] add qwen-vl-plus by @jugaldb in #12829
- [Azure OpenAI Feature] - Support DefaultAzureCredential without hard-coded environment variables by @ishaan-jaff in #12841
- Docs - litellm benchmarks by @ishaan-jaff in #12842
- fix(watsonx): IBM Watsonx - use correct parameter name for tool choice by @Tomas2D in #9980
- feat: add Hyperbolic provider support by @colesmcintosh in #12826
- Litellm batch cost tracking debug by @krrishdholakia in #12782
- Passthrough Auth - make Auth checks OSS + Anthropic - only show 'reasoning_effort' for supported models by @krrishdholakia in #12847
- build(deps): bump form-data from 4.0.0 to 4.0.4 in /ui/litellm-dashboard by @dependabot[bot] in #12851
- Fix team_member_budget update logic by @krrishdholakia in #12843
- Fix async retryer on
.acompletion()
+ forward clientside headers - filter outcontent-type
from clientside request (causes llm api call to hang) by @krrishdholakia in #12848 - Openrouter - filter out cache_control flag for non-anthropic models (allows usage with claude code) by @krrishdholakia in #12850
- Improvements on the Regenerate Key Flow by @NANDINI-star in #12788
- Replace non-root Dockerfile base with Alpine multi-stage build; by @mdiloreto in #12707
- build(deps): bump form-data from 4.0.3 to 4.0.4 in /docs/my-website by @dependabot[bot] in #12867
- Bug fix - Azure KeyVault not in image, add
azure-keyvault==4.2.0
to Docker img by @ishaan-jaff in #12873 - [Separate Health App] Pass through cmd args via supervisord by @jugaldb in #12871
- [Feat] Add Recraft API - Image Edits Support by @ishaan-jaff in #12874
- [LLM Translation] Litellm gemini 2.0 live support by @jugaldb in #12839
- [Feat] Add cost tracking for new vertex_ai/llama-3 API models by @ishaan-jaff in #12878
- [Feat] - Track cost + add tags for health checks done by LiteLLM Proxy by @ishaan-jaff in #12880
- Fix async retryer on .acompletion() + forward clientside headers - filter out content-type from clientside request (causes llm api call to hang) by @krrishdholakia in #12886
- Request Headers - support
x-litellm-num-retries
+ Usage - support usage by model group by @krrishdholakia in #12890 - [Docs ]Litellm mcp access group doc by @jugaldb in #12883
- [Add health check] add architecture diagram by @jugaldb in #12879
- rm retired anthropic models from model_prices_and_context_window.json by @staeiou in #12864
- feat: extended
/v1/models
endpoint, now it returns with fallbacks on demand by @murad-khafizov in #12811 - feat: Add Pillar Security guardrail integration by @eagle-p in #12791
- Swagger - bump DOMPurify version + add back Support local swagger bundle (good for airgapped environments) + support local swagger on custom root paths by @krrishdholakia in #12911
- [Feat] LiteLLM x Cloudzero integration - Allow exporting spend to cloudzero by @ishaan-jaff in #12908
- [Feat] LiteLLM CloudZero Integration updates - using LiteLLM_SpendLogs Table by @ishaan-jaff in #12922
- Add GA version of gemini 2.5 flash lite for both vertex and gemini by @apoorv-sharma in #12920
- [UI] Allow setting up CloudZero Usage through LiteLLM UI by @ishaan-jaff in #12923
- Update control_plane_and_data_plane.md by @superpoussin22 in #12939
- Prometheus - tags, fix '[tag]="false"' when tag is set by @krrishdholakia in #12916
- Proxy - specify
key_type
- allows specifying if key can call LLM API routes vs. Management routes only by @krrishdholakia in #12909 - fix(internal_user_endpoints.py): delete member from team table on
/user/delete
by @krrishdholakia in #12926 - [Feat] Backend Router - Add Auto-Router powered by
semantic-router
by @ishaan-jaff in #12955 - [LLM Translation] - Bug fix Anthropic Tool calling by @ishaan-jaff in #12959
- [LLM Translation] added new realtime model for openai by @jugaldb in #12946
- [Feat] UI - Allow Adding LiteLLM Auto Router on UI by @ishaan-jaff in #12960
- [Feat] Edit Auto Router Settings on UI by @ishaan-jaff in #12966
- [Feat] Add inpainting support and corresponding tests for Amazon Nova⦠by @SantoshDhaladhuli in #12949
- docs: added documentation about metadata exposed over the
/v1/models
endpoint by @murad-khafizov in #12942 - Fix: Shorten Gemini tool_call_id for Open AI compatibility by @tonga54 in #12941
- feat: add openrouter/qwen/qwen3-coder model configuration by @colesmcintosh in #12910
- fix(auth_utils): make header comparison case-insensitive by @sings-to-bees-on-wednesdays in #12950
- GuardrailsAI: use validatedOutput to allow usage of "fix" guards by @DmitriyAlergant in #12891
- Show global retry policy on UI by @krrishdholakia in #12969
- Guardrails - support model-level guardrails by @krrishdholakia in #12968
- build(github/manual_pypi_publish.yml): manual workflow to publish pip package - used for pushing dev releases by @krrishdholakia in #12985
- [LLM Translation] Add bytedance/ui-tars-1.5-7b on openrouter by @jugaldb in #12882
- clean and verify key before inserting by @jugaldb in #12840
- [LLM Translation] fix query params for realtime api intent by @jugaldb in #12838
- [MCP Gateway] Move cost tracking and permission management to OSS by @jugaldb in #12988
- build: update pip package by @krrishdholakia in #12998
- Fix issue writing db by @emerzon in #13001
- [BUG Fix] Cannot pickle coroutine object by @jugaldb in #13005
- [MCP Gateway] add Litellm mcp alias for prefixing by @jugaldb in #12994
- Fixup ollama model listing (again) by @matthid in #13008
- [Vector Store] make vector store permission management OSS by @jugaldb in #12990
- [FEAT] Model-Guardrails: Add on UI by @krrishdholakia in #13006
- UI SSO - fix reset env var when ui_access_mode is updated by @krrishdholakia in #13011
- [MCP Gateway] Litellm mcp multi header propagation by @jugaldb in #13003
New Contributors
- @magicalne made their first contribution in h...
v1.74.7-stable.patch.2
Full Changelog: v1.74.7-stable.patch.1...v1.74.7-stable.patch.2
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.74.7-stable.patch.2
Don't want to maintain your internal proxy? get in touch π
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.74.7-stable.patch.2
Don't want to maintain your internal proxy? get in touch π
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.74.7-stable.patch.2
Don't want to maintain your internal proxy? get in touch π
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
---|---|---|---|---|---|---|---|---|---|
/chat/completions | Passed β | 120.0 | 157.13796241417324 | 6.362622204908533 | 0.0 | 1905 | 0 | 82.22530700004427 | 1211.5445880000095 |
Aggregated | Passed β | 120.0 | 157.13796241417324 | 6.362622204908533 | 0.0 | 1905 | 0 | 82.22530700004427 | 1211.5445880000095 |
v1.74.7-stable.patch.1
Full Changelog: v1.74.7-stable...v1.74.7-stable.patch.1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.74.7-stable.patch.1
Don't want to maintain your internal proxy? get in touch π
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
---|---|---|---|---|---|---|---|---|---|
/chat/completions | Passed β | 170.0 | 210.81530860833013 | 6.340641152265373 | 0.0 | 1897 | 0 | 139.2534420000402 | 1204.8251080000227 |
Aggregated | Passed β | 170.0 | 210.81530860833013 | 6.340641152265373 | 0.0 | 1897 | 0 | 139.2534420000402 | 1204.8251080000227 |
v1.74.7-stable
Full Changelog: v1.74.7.rc.1...v1.74.7-stable
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.74.7-stable
Don't want to maintain your internal proxy? get in touch π
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
---|---|---|---|---|---|---|---|---|---|
/chat/completions | Passed β | 120.0 | 155.293356068966 | 6.395530747523885 | 0.0 | 1914 | 0 | 79.63155700002744 | 1028.7689699999873 |
Aggregated | Passed β | 120.0 | 155.293356068966 | 6.395530747523885 | 0.0 | 1914 | 0 | 79.63155700002744 | 1028.7689699999873 |
v1.74.8-nightly
What's Changed
- Fix SSO Logout | Create Unified Login Page with SSO and Username/Password Options by @NANDINI-star in #12703
- docs(moonshot): correct base url and document CN-specific endpoint by @magicalne in #12804
- Adding HolmesGPT to projects using LiteLLM by @pavangudiwada in #12798
- feat: Add Lambda AI provider support by @colesmcintosh in #12817
- [UI Bug Fix] Show correct guardrails when editing a team by @ishaan-jaff in #12823
- [Docs] Show correct list of vertex ai mistral models by @ishaan-jaff in #12828
- [LLM Translation - GH Copilot] added dynamic endpoint support by @jugaldb in #12827
- feat: add Morph provider support by @colesmcintosh in #12821
- fix: remove deprecated groq/qwen-qwq-32b and add qwen/qwen3-32b by @aholmberg in #12831
- Add Google Cloud Model Armor guardrail documentation by @colesmcintosh in #12814
- [Feat] Add Recraft Image Generation API Support - New LLM Provider by @ishaan-jaff in #12832
- [Bug Fix] - gemini leaking FD for sync calls with litellm.completion by @ishaan-jaff in #12824
- [Feat] Add fireworks -
fireworks/models/kimi-k2-instruct
by @ishaan-jaff in #12837 - [LLM Translation] add qwen-vl-plus by @jugaldb in #12829
- [Azure OpenAI Feature] - Support DefaultAzureCredential without hard-coded environment variables by @ishaan-jaff in #12841
- Docs - litellm benchmarks by @ishaan-jaff in #12842
- fix(watsonx): IBM Watsonx - use correct parameter name for tool choice by @Tomas2D in #9980
- feat: add Hyperbolic provider support by @colesmcintosh in #12826
- Litellm batch cost tracking debug by @krrishdholakia in #12782
- Passthrough Auth - make Auth checks OSS + Anthropic - only show 'reasoning_effort' for supported models by @krrishdholakia in #12847
- build(deps): bump form-data from 4.0.0 to 4.0.4 in /ui/litellm-dashboard by @dependabot[bot] in #12851
- Fix team_member_budget update logic by @krrishdholakia in #12843
- Fix async retryer on
.acompletion()
+ forward clientside headers - filter outcontent-type
from clientside request (causes llm api call to hang) by @krrishdholakia in #12848 - Openrouter - filter out cache_control flag for non-anthropic models (allows usage with claude code) by @krrishdholakia in #12850
- Improvements on the Regenerate Key Flow by @NANDINI-star in #12788
- Replace non-root Dockerfile base with Alpine multi-stage build; by @mdiloreto in #12707
- build(deps): bump form-data from 4.0.3 to 4.0.4 in /docs/my-website by @dependabot[bot] in #12867
- Bug fix - Azure KeyVault not in image, add
azure-keyvault==4.2.0
to Docker img by @ishaan-jaff in #12873 - [Separate Health App] Pass through cmd args via supervisord by @jugaldb in #12871
- [Feat] Add Recraft API - Image Edits Support by @ishaan-jaff in #12874
- [LLM Translation] Litellm gemini 2.0 live support by @jugaldb in #12839
- [Feat] Add cost tracking for new vertex_ai/llama-3 API models by @ishaan-jaff in #12878
- [Feat] - Track cost + add tags for health checks done by LiteLLM Proxy by @ishaan-jaff in #12880
- Fix async retryer on .acompletion() + forward clientside headers - filter out content-type from clientside request (causes llm api call to hang) by @krrishdholakia in #12886
- Request Headers - support
x-litellm-num-retries
+ Usage - support usage by model group by @krrishdholakia in #12890 - [Docs ]Litellm mcp access group doc by @jugaldb in #12883
- [Add health check] add architecture diagram by @jugaldb in #12879
- rm retired anthropic models from model_prices_and_context_window.json by @staeiou in #12864
- feat: extended
/v1/models
endpoint, now it returns with fallbacks on demand by @murad-khafizov in #12811 - feat: Add Pillar Security guardrail integration by @eagle-p in #12791
- Swagger - bump DOMPurify version + add back Support local swagger bundle (good for airgapped environments) + support local swagger on custom root paths by @krrishdholakia in #12911
- [Feat] LiteLLM x Cloudzero integration - Allow exporting spend to cloudzero by @ishaan-jaff in #12908
New Contributors
- @magicalne made their first contribution in #12804
- @pavangudiwada made their first contribution in #12798
- @mdiloreto made their first contribution in #12707
- @murad-khafizov made their first contribution in #12811
- @eagle-p made their first contribution in #12791
Full Changelog: v1.74.7.rc.1...v1.74.8-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.74.8-nightly
Don't want to maintain your internal proxy? get in touch π
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
---|---|---|---|---|---|---|---|---|---|
/chat/completions | Passed β | 170.0 | 208.0958998087373 | 6.271169991247407 | 0.0 | 1877 | 0 | 130.56155500004252 | 1168.738483000027 |
Aggregated | Passed β | 170.0 | 208.0958998087373 | 6.271169991247407 | 0.0 | 1877 | 0 | 130.56155500004252 | 1168.738483000027 |
v1.74.3-stable.patch.4
Full Changelog: v1.74.3-stable.patch.3...v1.74.3-stable.patch.4
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.74.3-stable.patch.4
Don't want to maintain your internal proxy? get in touch π
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
---|---|---|---|---|---|---|---|---|---|
/chat/completions | Passed β | 160.0 | 204.61626603598248 | 6.22322538904882 | 0.0 | 1862 | 0 | 127.11543199998232 | 1627.7963420000106 |
Aggregated | Passed β | 160.0 | 204.61626603598248 | 6.22322538904882 | 0.0 | 1862 | 0 | 127.11543199998232 | 1627.7963420000106 |
v1.72.2-stable.debug
Full Changelog: v1.72.2-stable...v1.72.2-stable.debug
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.72.2-stable.debug
Don't want to maintain your internal proxy? get in touch π
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
---|---|---|---|---|---|---|---|---|---|
/chat/completions | Passed β | 130.0 | 168.6584858474577 | 6.307054920240884 | 0.0 | 1888 | 0 | 95.40891199998214 | 1772.0808789999865 |
Aggregated | Passed β | 130.0 | 168.6584858474577 | 6.307054920240884 | 0.0 | 1888 | 0 | 95.40891199998214 | 1772.0808789999865 |