Memory Leak in chat completion create

[rdy5644]

Confirm this is an issue with the Python library and not an underlying OpenAI API

• This is an issue with the Python library

Describe the bug

After Making many streaming calls from a flask app the process memory is infinitely increasing and it is never getting reduced.
Even after performing close on response and client.

To Reproduce

1. Wrap the below code in flask api or you can run as a long running process as well.
2. Invoke this method in a loop or for 1000 you will be able to see the process gradually takes up more memory.
3. Even performing manual gc didn't help.

Code snippets

def make_stream_call_with_close():
    client = OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))
    responses = client.chat.completions.create(
        model='gpt-3.5-turbo',
        messages=[{"role": "user", "content": "Say exactly one word."}],
        stream=True,
    )
    responses.response.close()
    print(responses.response.is_closed)
    client.close()
    print(client.is_closed())

OS

macOS,Linux

Python version

3.10.13

Library version

openai==1.3.4 , httpx==0.27.0 , pydantic==2.7.1, pydantic_core==2.18.2

[rdy5644]

tracemalloc top 10 after performing few requests

Snapshot comparison before invoke and after invoking once 1.
00:00:00 Top 10 differences: /Users/jogireddy/PycharmProjects/flaskProject/.venv/lib/python3.10/site-packages/openai/_response.py:238: size=61.4 KiB (+61.4 KiB), count=525 (+525), average=120 B/usr/local/Cellar/python@3.10/3.10.13/Frameworks/Python.framework/Versions/3.10/lib/python3.10/abc.py:123: size=54.1 KiB (+54.1 KiB), count=614 (+614), average=90 B<frozen importlib._bootstrap_external>:672: size=94.6 KiB (+51.6 KiB), count=1546 (+570), average=63 B<frozen importlib._bootstrap>:241: size=30.6 KiB (+30.6 KiB), count=316 (+316), average=99 B/usr/local/Cellar/python@3.10/3.10.13/Frameworks/Python.framework/Versions/3.10/lib/python3.10/functools.py:52: size=21.4 KiB (+21.4 KiB), count=106 (+106), average=207 B/usr/local/Cellar/python@3.10/3.10.13/Frameworks/Python.framework/Versions/3.10/lib/python3.10/functools.py:58: size=15.8 KiB (+15.8 KiB), count=239 (+239), average=68 B/usr/local/Cellar/python@3.10/3.10.13/Frameworks/Python.framework/Versions/3.10/lib/python3.10/typing.py:694: size=14.8 KiB (+14.8 KiB), count=32 (+32), average=475 B/Users/jogireddy/PycharmProjects/flaskProject/.venv/lib/python3.10/site-packages/openai/_resource.py:22: size=10184 B (+10184 B), count=70 (+70), average=145 B/usr/local/Cellar/python@3.10/3.10.13/Frameworks/Python.framework/Versions/3.10/lib/python3.10/typing.py:989: size=4464 B (+4264 B), count=62 (+59), average=72 B/usr/local/Cellar/python@3.10/3.10.13/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/cookiejar.py:964: size=2720 B (+2720 B), count=6 (+6), average=453 B

Snapshot compare before invoking and after invoking for 8-10 times in loop
00:00:00 Top 10 differences: /Users/jogireddy/PycharmProjects/flaskProject/.venv/lib/python3.10/site-packages/openai/_response.py:238: size=899 KiB (+899 KiB), count=7731 (+7731), average=119 B/usr/local/Cellar/python@3.10/3.10.13/Frameworks/Python.framework/Versions/3.10/lib/python3.10/functools.py:58: size=190 KiB (+190 KiB), count=2815 (+2815), average=69 B/usr/local/Cellar/python@3.10/3.10.13/Frameworks/Python.framework/Versions/3.10/lib/python3.10/tracemalloc.py:115: size=126 KiB (+126 KiB), count=1615 (+1615), average=80 B/usr/local/Cellar/python@3.10/3.10.13/Frameworks/Python.framework/Versions/3.10/lib/python3.10/abc.py:123: size=52.7 KiB (+52.7 KiB), count=591 (+591), average=91 B<frozen importlib._bootstrap_external>:672: size=94.4 KiB (+51.4 KiB), count=1545 (+569), average=63 B/Users/jogireddy/PycharmProjects/flaskProject/.venv/lib/python3.10/site-packages/openai/_resource.py:17: size=50.0 KiB (+50.0 KiB), count=696 (+696), average=73 B/Users/jogireddy/PycharmProjects/flaskProject/.venv/lib/python3.10/site-packages/openai/_resource.py:22: size=30.9 KiB (+30.9 KiB), count=406 (+406), average=78 B<frozen importlib._bootstrap>:241: size=30.6 KiB (+30.6 KiB), count=315 (+315), average=99 B/Users/jogireddy/PycharmProjects/flaskProject/.venv/lib/python3.10/site-packages/openai/_resource.py:23: size=22.5 KiB (+22.5 KiB), count=360 (+360), average=64 B/Users/jogireddy/PycharmProjects/flaskProject/.venv/lib/python3.10/site-packages/openai/_resource.py:21: size=22.5 KiB (+22.5 KiB), count=360 (+360), average=64 B

Snapshot before invoking and after invoking for 10-15 times

/Users/jogireddy/PycharmProjects/flaskProject/.venv/lib/python3.10/site-packages/openai/_response.py:238: size=1112 KiB (+216 KiB), count=9255 (+1541), average=123 B/usr/local/Cellar/python@3.10/3.10.13/Frameworks/Python.framework/Versions/3.10/lib/python3.10/functools.py:58: size=239 KiB (+49.7 KiB), count=3550 (+736), average=69 B/Users/jogireddy/PycharmProjects/flaskProject/.venv/lib/python3.10/site-packages/openai/_resource.py:17: size=63.8 KiB (+13.8 KiB), count=888 (+192), average=74 B/Users/jogireddy/PycharmProjects/flaskProject/.venv/lib/python3.10/site-packages/openai/_resource.py:22: size=36.9 KiB (+6144 B), count=502 (+96), average=75 B/Users/jogireddy/PycharmProjects/flaskProject/.venv/lib/python3.10/site-packages/openai/_resource.py:23: size=28.5 KiB (+6144 B), count=456 (+96), average=64 B/Users/jogireddy/PycharmProjects/flaskProject/.venv/lib/python3.10/site-packages/openai/_resource.py:21: size=28.5 KiB (+6144 B), count=456 (+96), average=64 B/Users/jogireddy/PycharmProjects/flaskProject/.venv/lib/python3.10/site-packages/openai/_resource.py:20: size=28.5 KiB (+6144 B), count=456 (+96), average=64 B/Users/jogireddy/PycharmProjects/flaskProject/.venv/lib/python3.10/site-packages/openai/_resource.py:19: size=28.5 KiB (+6144 B), count=456 (+96), average=64 B/Users/jogireddy/PycharmProjects/flaskProject/.venv/lib/python3.10/site-packages/openai/_resource.py:18: size=28.5 KiB (+6144 B), count=456 (+96), average=64 B/Users/jogireddy/PycharmProjects/flaskProject/.venv/lib/python3.10/site-packages/pydantic/_internal/_core_utils.py:200: size=3952 B (+3784 B), count=21 (+20), average=188 B

The Process memory usage went from 80M to 326M after invoking this method for more then 200 times.

[rdy5644]

Found this function at openai/_response.py:238

@functools.wraps(func) def wrapped(*args: P.args, **kwargs: P.kwargs) -> APIResponse[R]: extra_headers = {**(cast(Any, kwargs.get("extra_headers")) or {})} extra_headers[RAW_RESPONSE_HEADER] = "true" kwargs["extra_headers"] = extra_headers return cast(APIResponse[R], func(*args, **kwargs)) return wrapped

[rattrayalex]

Thanks, we'll look into this!

[RobertCraigie]

Please upgrade to the latest version, you're quite far behind and we've fixed a bunch of memory leak issues since then. If this is still occurring then please re-open this issue.

[rdy5644]

I upgraded openai version to 1.23.5

and still I can see some issues with the same function just it got moved to different place.
/Users/jogireddy/PycharmProjects/flaskProject/venv/lib/python3.9/site-packages/openai/_legacy_response.py:347: size=1389 KiB (+1389 KiB), count=12109 (+12109), average=117 B/Users/jogireddy/PycharmProjects/flaskProject/venv/lib/python3.9/site-packages/openai/_response.py:674: size=1278 KiB (+1278 KiB), count=11239 (+11239), average=116 B/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/linecache.py:137: size=1232 KiB (+1232 KiB), count=12014 (+12014), average=105 B/Users/jogireddy/PycharmProjects/flaskProject/venv/lib/python3.9/site-packages/httpx/_content.py:175: size=868 KiB (+868 KiB), count=83 (+83), average=10.5 KiB/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/functools.py:58: size=769 KiB (+769 KiB), count=12308 (+12308), average=64 B/Users/jogireddy/PycharmProjects/flaskProject/venv/lib/python3.9/site-packages/openai/_legacy_response.py:330: size=726 KiB (+726 KiB), count=6332 (+6332), average=117 B/Users/jogireddy/PycharmProjects/flaskProject/venv/lib/python3.9/site-packages/openai/_response.py:653: size=676 KiB (+676 KiB), count=5931 (+5931), average=117 B/Users/jogireddy/PycharmProjects/flaskProject/venv/lib/python3.9/site-packages/openai/_resource.py:34: size=358 KiB (+358 KiB), count=5088 (+5088), average=72 B/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/abc.py:102: size=261 KiB (+261 KiB), count=2992 (+2992), average=89 B/Users/jogireddy/PycharmProjects/flaskProject/venv/lib/python3.9/site-packages/httpx/_models.py:82: size=249 KiB (+249 KiB), count=3736 (+3736), average=68 B