v1.44.21-stable

What's Changed

• [Fix] OTEL - Unsupported | type annotations in python3.9 by @ishaan-jaff in #5548
• Update Hugging Face Doc by @gary149 in #5411
• [Fix-Datdog Logger] Log exceptions when callbacks faces an error by @ishaan-jaff in #5570
• fix(navbar.tsx): only show 'get enterprise license' if user is not already a premium user by @krrishdholakia in #5568
• LiteLLM Minor Fixes and Improvements (08/06/2024) by @krrishdholakia in #5567
• [Feat-Proxy] Use DB Views to Get spend per Tag (Usage endpoints) by @ishaan-jaff in #5574
• [Feat] Allow setting duration time when regenerating key by @ishaan-jaff in #5566
• [Feat] Add cost tracking for cohere rerank by @ishaan-jaff in #5573
• Allow client-side credentials to be sent to proxy (accept only if complete credentials are given) by @krrishdholakia in #5575

New Contributors

• @gary149 made their first contribution in #5411

Full Changelog: v1.44.19...v1.44.21-stable

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.44.21-stable

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	110.0	137.28568682202445	6.405426266891142	0.0	1916	0	96.66716199996017	1538.306079999984
Aggregated	Passed ✅	110.0	137.28568682202445	6.405426266891142	0.0	1916	0	96.66716199996017	1538.306079999984

v1.44.21

What's Changed

• [Fix] OTEL - Unsupported | type annotations in python3.9 by @ishaan-jaff in #5548
• Update Hugging Face Doc by @gary149 in #5411
• [Fix-Datdog Logger] Log exceptions when callbacks faces an error by @ishaan-jaff in #5570
• fix(navbar.tsx): only show 'get enterprise license' if user is not already a premium user by @krrishdholakia in #5568
• LiteLLM Minor Fixes and Improvements (08/06/2024) by @krrishdholakia in #5567
• [Feat-Proxy] Use DB Views to Get spend per Tag (Usage endpoints) by @ishaan-jaff in #5574
• [Feat] Allow setting duration time when regenerating key by @ishaan-jaff in #5566
• [Feat] Add cost tracking for cohere rerank by @ishaan-jaff in #5573
• Allow client-side credentials to be sent to proxy (accept only if complete credentials are given) by @krrishdholakia in #5575

New Contributors

• @gary149 made their first contribution in #5411

Full Changelog: v1.44.19...v1.44.21

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.44.21

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	130.0	166.20606541064828	6.339987548576783	0.0	1897	0	112.93849100002262	2116.0177159999876
Aggregated	Passed ✅	130.0	166.20606541064828	6.339987548576783	0.0	1897	0	112.93849100002262	2116.0177159999876

v1.44.19-stable

Excited to Launch the ability for LiteLLM Gateway (Proxy) Users to - Create Virtual Keys for 100+ LLMs and track their own Usage Start here: https://github.com/BerriAI/litellm/releases/tag/v1.44.19-stable

✨ [UI] Show when a virtual key expires

✨ UI - show expired virtual keys on Admin UI

What's Changed

• fix KeyError when call deepseek api by @wolf-joe in #5530
• [UI] Show when a virtual key expires by @ishaan-jaff in #5541
• [Fix-Proxy] allow internal user and internal viewer to view usage by @ishaan-jaff in #5536
• LiteLLM Merged PR's by @krrishdholakia in #5538
• Update lago.py to accomodate API change (#5495) by @krrishdholakia in #5543
• LiteLLM Minor Fixes and Improvements by @krrishdholakia in #5537
• [Fix] transcription/atranscription file parameter should accept correct types by @ishaan-jaff in #5534

New Contributors

• @wolf-joe made their first contribution in #5530

Full Changelog: v1.44.18...v1.44.19-stable

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.44.19-stable

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	84	98.93160195485267	6.438868020860198	0.0	1927	0	68.17092200003572	1581.8675439999765
Aggregated	Passed ✅	84	98.93160195485267	6.438868020860198	0.0	1927	0	68.17092200003572	1581.8675439999765

v1.44.19

What's Changed

• fix KeyError when call deepseek api by @wolf-joe in #5530
• [UI] Show when a virtual key expires by @ishaan-jaff in #5541
• [Fix-Proxy] allow internal user and internal viewer to view usage by @ishaan-jaff in #5536
• LiteLLM Merged PR's by @krrishdholakia in #5538
• Update lago.py to accomodate API change (#5495) by @krrishdholakia in #5543
• LiteLLM Minor Fixes and Improvements by @krrishdholakia in #5537
• [Fix] transcription/atranscription file parameter should accept correct types by @ishaan-jaff in #5534

New Contributors

• @wolf-joe made their first contribution in #5530

Full Changelog: v1.44.18...v1.44.19

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.44.19

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	140.0	169.38227208086795	6.321167248443861	0.0	1892	0	114.4958109999834	2909.539383999999
Aggregated	Passed ✅	140.0	169.38227208086795	6.321167248443861	0.0	1892	0	114.4958109999834	2909.539383999999

v1.44.18-stable

What's Changed

• LiteLLM Minor fixes + improvements (08/04/2024) by @krrishdholakia in #5505
• fix(pass_through_endpoints): support bedrock agents via pass through by @krrishdholakia in #5527

Full Changelog: v1.44.17...v1.44.18-stable

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.44.18-stable

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	150.0	170.81621034830297	6.398015939395273	0.0	1915	0	116.37449199997718	2974.9741289999747
Aggregated	Passed ✅	150.0	170.81621034830297	6.398015939395273	0.0	1915	0	116.37449199997718	2974.9741289999747

v1.44.18

What's Changed

• LiteLLM Minor fixes + improvements (08/04/2024) by @krrishdholakia in #5505
• fix(pass_through_endpoints): support bedrock agents via pass through by @krrishdholakia in #5527

Full Changelog: v1.44.17...v1.44.18

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.44.18

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	110.0	123.78989845407648	6.4836946715014685	0.0	1938	0	82.95310500000141	2277.6017620000175
Aggregated	Passed ✅	110.0	123.78989845407648	6.4836946715014685	0.0	1938	0	82.95310500000141	2277.6017620000175

v1.44.17-stable

What's Changed

• Add azure/gpt-4o-2024-08-06 pricing. by @Manouchehri in #5510
• [Fix] get_llm_provider, return provider as cohere_chat for cohere chat models by @ishaan-jaff in #5511
• fix proxy server - always read redis for rate limiting logic by @ishaan-jaff in #5484
• Helicone stream response logging by @maamalama in #5516
• security - Prevent sql injection in /team/update query by @krrishdholakia in #5513
• [Fix-Refactor] support presidio on new guardrails config by @ishaan-jaff in #5514
• [Fix - Proxy] show error from /spend/tags and /spend/logs on client side by @ishaan-jaff in #5520
• [Feat] log request / response on pass through endpoints by @ishaan-jaff in #5518
• [Fix-Proxy] show more descriptive error messages on /health checks by @ishaan-jaff in #5521

Full Changelog: v1.44.16...v1.44.17-stable

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.44.17-stable

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	150.0	168.624452710735	6.331403580740946	0.0	1891	0	110.9297180000226	815.6032489999916
Aggregated	Passed ✅	150.0	168.624452710735	6.331403580740946	0.0	1891	0	110.9297180000226	815.6032489999916

v1.44.17

What's Changed

• Add azure/gpt-4o-2024-08-06 pricing. by @Manouchehri in #5510
• [Fix] get_llm_provider, return provider as cohere_chat for cohere chat models by @ishaan-jaff in #5511
• fix proxy server - always read redis for rate limiting logic by @ishaan-jaff in #5484
• Helicone stream response logging by @maamalama in #5516
• security - Prevent sql injection in /team/update query by @krrishdholakia in #5513
• [Fix-Refactor] support presidio on new guardrails config by @ishaan-jaff in #5514
• [Fix - Proxy] show error from /spend/tags and /spend/logs on client side by @ishaan-jaff in #5520
• [Feat] log request / response on pass through endpoints by @ishaan-jaff in #5518
• [Fix-Proxy] show more descriptive error messages on /health checks by @ishaan-jaff in #5521

Full Changelog: v1.44.16...v1.44.17

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.44.17

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	110.0	144.75345275454978	6.424739460241986	0.0	1923	0	86.76964199997883	3621.227346000012
Aggregated	Passed ✅	110.0	144.75345275454978	6.424739460241986	0.0	1923	0	86.76964199997883	3621.227346000012

v1.44.16-stable

What's Changed

• update canary by @yujonglee in #5459
• Bump pagefind from 1.1.0 to 1.1.1 in /docs/my-website by @dependabot in #5491
• [Feat] Add Google Secret Manager Support by @ishaan-jaff in #5489
• [Feat-Proxy] Enterprise - allow controlling allowed private, public, admin only routes by @ishaan-jaff in #5494
• [Feat-Proxy] bump langfuse sdk version on docker by @ishaan-jaff in #5497
• LiteLLM Minor fixes + improvements (08/03/2024) by @krrishdholakia in #5488

Full Changelog: v1.44.15...v1.44.16-stable

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.44.16-stable

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	140.0	168.79004244876447	6.3642503777567425	0.0	1903	0	113.65976999996974	2153.1978849999405
Aggregated	Passed ✅	140.0	168.79004244876447	6.3642503777567425	0.0	1903	0	113.65976999996974	2153.1978849999405

v1.44.16

What's Changed

• update canary by @yujonglee in #5459
• Bump pagefind from 1.1.0 to 1.1.1 in /docs/my-website by @dependabot in #5491
• [Feat] Add Google Secret Manager Support by @ishaan-jaff in #5489
• [Feat-Proxy] Enterprise - allow controlling allowed private, public, admin only routes by @ishaan-jaff in #5494
• [Feat-Proxy] bump langfuse sdk version on docker by @ishaan-jaff in #5497
• LiteLLM Minor fixes + improvements (08/03/2024) by @krrishdholakia in #5488

Full Changelog: v1.44.15...v1.44.16

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.44.16

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	150.0	172.57036657744393	6.428433196552351	0.0033411814950895795	1924	1	111.12589300000764	1407.5345470000116
Aggregated	Passed ✅	150.0	172.57036657744393	6.428433196552351	0.0033411814950895795	1924	1	111.12589300000764	1407.5345470000116