v1.73.6.rc.1
What's Changed
- [⚡️ Python SDK Import] - 2 second faster import times by @ishaan-jaff in #12135
- 🧹 Refactor init.py to use a model registry by @ishaan-jaff in #12138
- Revert "🧹 Refactor init.py to use a model registry" by @ishaan-jaff in #12141
- [⚡️ Python SDK import] - reduce python sdk import time by .3s by @ishaan-jaff in #12140
/v1/messages- Remove hardcoded model name on streaming + Tags - enable setting custom header tags by @krrishdholakia in #12131- UI QA Fixes - prevent team model reset on model add + return team-only models on /v2/model/info + render team member budget correctly by @krrishdholakia in #12144
Full Changelog: v1.73.6-nightly...v1.73.6.rc.1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.73.6.rc.1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 190.0 | 207.64485716560858 | 6.320692412920188 | 0.0 | 1890 | 0 | 167.53384199989796 | 1605.5699650000292 |
| Aggregated | Passed ✅ | 190.0 | 207.64485716560858 | 6.320692412920188 | 0.0 | 1890 | 0 | 167.53384199989796 | 1605.5699650000292 |
Contributors
krrishdholakia and ishaan-jaff