new sessions evert 3 minutes

[ShantanuNair]

Describe the bug

https://github.com/openai/openai-python/blob/e389823ba013a24b4c32ce38fa0bd87e6bccae94/openai/api_requestor.py#L590C2-L590C2

This MAX_SESSION_LIFETIME_SECS setting kills sessions after 3 minutes. I believe this breaks long running tasks. It would also be difficult to troubleshoot downstream issues if the client is killing sessions at an arbitrary 3 minutes. Can someone verify the PR the brought this change is 100% good to go? It seems to me like this shouldn't be there, or at least be configurable, and that it would mess with users passing in sessions.

elif (
            time.time() - getattr(_thread_context, "session_create_time", 0)
            >= MAX_SESSION_LIFETIME_SECS
        ):
            _thread_context.session.close()
            _thread_context.session = _make_session()
            _thread_context.session_create_time = time.time()

@jhallard

To Reproduce

Run a long running llm request, and a new session is created after 3 minutes during.

Code snippets

No response

OS

macOS

Python version

3.8

Library version

openai-python v0.28.0

[jhallard]

Hello @ShantanuNair, the fix to periodically close sessions did fix the originally reported issue of leaking sessions in certain contexts. Is the issue here that you're

1. Providing a session to the OAI client
2. Making a long-running request (that runs for longer than 3 minutes)
3. Making a second request in parallel which triggers this logic to reset the session
4. The original long-running request is now killed halfway through?

[RobertCraigie]

This is resolved in our new, fully rewritten v1.0.0 library, now in beta, as we now support a class instance based API:

from openai import OpenAI

client = OpenAI(
  api_key=os.environ['OPENAI_API_KEY'],  # this is also the default, it can be omitted
)

client.chat.completions.create(...)

Please give it a try, and share feedback (big or small) in the linked discussion! (Not in this thread).

[ansumanswain]

fixed.