Fix: Properly close aiohttp client sessions to prevent resource leaks

[colesmcintosh]

Title

Fix: Properly close aiohttp client sessions to prevent resource leaks

Relevant issues

Fixes #12107

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
• I have added a screenshot of my new test passing locally
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

🐛 Bug Fix

Changes

This PR fixes the resource leak issue reported in #12107 where aiohttp client sessions were not being properly closed when using acompletion with Gemini models (and other models that use aiohttp).

Problem

When using acompletion, the code creates aiohttp ClientSession objects that are cached but never closed, leading to "Unclosed client session" and "Unclosed connector" warnings.

Solution

1. Added close() method to BaseLLMAIOHTTPHandler: This method properly closes the aiohttp ClientSession if it exists and is not already closed.

2. Created async_client_cleanup module: This module provides utility functions to close all cached async clients:

   • close_litellm_async_clients(): Manually close all cached async clients
   • register_async_client_cleanup(): Register automatic cleanup at exit using atexit

3. Automatic cleanup registration: The cleanup is automatically registered when litellm is imported, ensuring clients are closed when the program exits.

4. Exported manual cleanup function: Users can also manually call litellm.close_litellm_async_clients() if they need to clean up resources before exit.

Testing

Added comprehensive tests in tests/test_resource_cleanup.py that verify:

• Single acompletion calls don't leave unclosed resources
• Multiple acompletion calls reuse clients and don't leak resources
• The cleanup function can be called multiple times safely

Test Results

$ source .venv/bin/activate && python -m pytest tests/test_resource_cleanup.py -v
============================= test session starts ==============================
platform darwin -- Python 3.11.13, pytest-7.4.4, pluggy-1.5.0 -- /Users/colesmcintosh/Projects/litellm/.venv/bin/python
cachedir: .pytest_cache
rootdir: /Users/colesmcintosh/Projects/litellm
plugins: xdist-3.7.0, respx-0.22.0, anyio-4.5.2, mock-3.14.1, asyncio-0.21.2, retry-1.6.3, requests-mock-1.12.1
asyncio: mode=Mode.STRICT
collecting ... collected 3 items

tests/test_resource_cleanup.py::test_acompletion_resource_cleanup PASSED [ 33%]
tests/test_resource_cleanup.py::test_multiple_acompletion_calls_cleanup PASSED [ 66%]
tests/test_resource_cleanup.py::test_cleanup_function_is_safe_to_call_multiple_times PASSED [100%]

======================== 3 passed, 2 warnings in 1.34s =========================

Usage Example

The workaround mentioned in the issue is now built-in:

import litellm
import asyncio

async def main(): 
    response = await litellm.acompletion(
        model="gemini/gemini-2.0-flash-lite-001", 
        messages=[{"role": "user", "content": "Hello"}],
    )
    print(response.choices[0].message.content)
    
    # Optional: manually clean up (also happens automatically at exit)
    await litellm.close_litellm_async_clients()

if __name__ == "__main__":
    asyncio.run(main())

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
litellm	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jul 8, 2025 10:20pm