v1.32.3

🚨 Nightly Build - We noticed testing was flaky on this release

https://docs.litellm.ai/docs/proxy/logging

LiteLLM v1.32.3 📈 Proxy 100+ LLMs in one format + Send logs to Datadog. Start here: https://docs.litellm.ai/docs/proxy/logging

👉 Admin UI: Bug Fix for viewing total spend on LiteLLM Proxy https://docs.litellm.ai/docs/proxy/ui

💵 New /global/spend endpoint -> get total spend on proxy, total proxy budget if set

📖 Docs fix - view Pre Call hooks https://docs.litellm.ai/docs/proxy/call_hooks

📖 Docs - Using LiteLLM Proxy + Datadog for sending LLM logs:
https://docs.litellm.ai/docs/proxy/logging#logging-proxy-inputoutput---datadog

What's Changed

• fix(caching.py): pass redis kwargs to connection pool init by @krrishdholakia in #2573
• Support multiple system message tranlation for bedrock claude-3 by @garfeildma in #2570
• [FEAT] DataDog Logging Provider by @ishaan-jaff in #2578
• [Docs] datadog docs + LiteLLM Proxy by @ishaan-jaff in #2579
• fix bug: custom prompt templates registered are never applied to vllm provider by @zdsfwy in #2548
• improving non-openai tool_call prompt by @TheDiscoMole in #1527
• feat(vertex_ai.py): support gemini (vertex ai) function calling when streaming by @krrishdholakia in #2577
• [FIX] UI show global spend - when proxy budget not set by @ishaan-jaff in #2581
• fix(proxy/utils.py): move to batch writing db updates by @krrishdholakia in #2561

New Contributors

• @garfeildma made their first contribution in #2570
• @zdsfwy made their first contribution in #2548
• @TheDiscoMole made their first contribution in #1527

Full Changelog: v1.32.1...v1.32.3

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	87	95.22148051583896	1.4766004734948956	0.0	442	0	81.41332799999645	1162.7792299999555
/health/liveliness	Passed ✅	62	65.23377382330264	15.597845273184769	0.006681450106311745	4669	2	59.51579199995649	1170.440439999993
/health/readiness	Passed ✅	62	65.01079326000487	15.110099415424012	0.0	4523	0	59.88310399993679	1250.393838999912
Aggregated	Passed ✅	62	66.5048995520034	32.184545162103674	0.006681450106311745	9634	2	59.51579199995649	1250.393838999912