[Feat] LiteLLM Allow setting Uvicorn Keep Alive Timeout

[ishaan-jaff]

[Feat] LiteLLM Allow setting Uvicorn Keep Alive Timeout

This PR adds support for configuring the Uvicorn keepalive timeout in LiteLLM

Relevant issues

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
• I have added a screenshot of my new test passing locally
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

🆕 New Feature
✅ Test

Changes

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
litellm	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jun 10, 2025 8:06pm

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ ishaan-jaff
❌ cursoragent
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[Copilot]

Pull Request Overview

This PR adds support for configuring the Uvicorn keepalive timeout in LiteLLM, enhancing the server configuration flexibility. Key changes include adding a new CLI option and associated parameter in the Uvicorn initialization helper, updating tests for the new parameter, and documenting the feature in deployment instructions.

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File	Description
tests/test_litellm/proxy/test_proxy_cli.py	Added tests to verify the keepalive_timeout flag and parameter usage.
litellm/proxy/proxy_cli.py	Introduced the keepalive_timeout parameter in CLI and helper methods.
docs/my-website/docs/proxy/deploy.md	Updated documentation with usage examples for keepalive_timeout.

Comments suppressed due to low confidence (1)

docs/my-website/docs/proxy/deploy.md:688

• The documentation specifies a default keepalive timeout of 5 seconds, but the CLI option default is None and no default value is set in the helper function. Please update the code to set a default of 5 seconds or modify the docs to reflect the current behavior.

Defaults to 5 seconds. Between requests, connections must receive new data within this period or be disconnected.