RecursionError in openai\_utils\_proxy.py

[xuing]

Confirm this is an issue with the Python library and not an underlying OpenAI API

• This is an issue with the Python library

Describe the bug

A RecursionError is encountered when debugging a Python script that includes import openai in PyCharm.

The specific error occurs within the openai library's _utils_proxy.py file, involving the get_proxied and getattr methods.

The error message is as follows:

 File "<venv_path>\Lib\site-packages\openai\_utils\_proxy.py", line 45, in __get_proxied__
    proxied = self.__proxied
  File "<venv_path>\Lib\site-packages\openai\_utils\_proxy.py", line 22, in __getattr__
    return getattr(self.__get_proxied__(), attr)
  ...
RecursionError: maximum recursion depth exceeded

This error is triggered upon initiating the PyCharm debugger to debug the script.

stack trace:

...
getattr, _proxy.py:22
get_proxied, _proxy.py:45
getattr, _proxy.py:22
get_proxied, _proxy.py:45
getattr, _proxy.py:22
get_proxied, _proxy.py:45
class, _proxy.py:39
init, _proxy.py:18
<module>, numpy_proxy.py:29
<module>, init.py:1
<module>, embeddings.py:14
<module>, init.py:20
<module>, _client.py:12
<module>, init.py:11
<module>, temp.py:1

To Reproduce

1. In PyCharm, create and open a Python script that imports the OpenAI library (import openai).
2. Configure PyCharm for debugging the script.
3. Start the debugger in PyCharm. The command executed by PyCharm will be similar to:
   <python_executable> -X pycache_prefix=<cache_directory> <pydevd_path> --multiprocess --qt-support --client <client_ip> --port <port> --file test.py
4. Observe the RecursionError during the debugging process when the OpenAI library is imported.

PyCharm 2023.3 EAP (Professional Edition)

Code snippets

import openai

print("1")

OS

Windows11

Python version

Python 3.12.0

Library version

1.3.6

[RobertCraigie]

Can you try installing the package from our next branch? This may have been fixed

[xuing]

Can you try installing the package from our next branch? This may have been fixed

perhaps not?

[RobertCraigie]

@xuing I think this might be a PyCharm bug, I can't reproduce the issue using 2023.2.2 (Professional Edition) but I can reproduce it using PyCharm 2023.3 EAP.

I can't see anything suspect in our code, the last stack before the recursion is this:

while frame and isinstance(frame.f_locals.get('self'), PEP669CallbackBase):

Coming from PyCharm/Contents/plugins/python/helpers/pydev/_pydevd_bundle/pydevd_pep_669_tracing.py.

While the proxy class is a little non-standard, the issue here is coming from the fact that we're overriding __getattr__ which I think is pretty common.

[xuing]

@RobertCraigie I think what you're saying makes a lot of sense.

It indeed might be an issue with pydev, considering it's an EAP version. I will switch versions to avoid this bug and then see if I can provide feedback to JetBrains or PyDev.Debugger

Thank you for your attempts and explanation.

By the way, the just-released RC version also has the same issue.

[arthurdedeus]

@xuing if you've opened a ticket in JetBrains support, can you send me the link to it? I am facing the exact same issue.
I can't debug when using the newer versions of PyCharm, but it worked fine when using 2023.2.5.

[xuing]

@xuing if you've opened a ticket in JetBrains support, can you send me the link to it? I am facing the exact same issue. I can't debug when using the newer versions of PyCharm, but it worked fine when using 2023.2.5.

https://youtrack.jetbrains.com/issue/PY-64445/RecursionError-with-OpenAI-Library-in-PyDev.Debugger-of-PyCharm-2023.3-RC

[Roman-Bober]

same issue again with latest paycharm
PyCharm 2023.3.2 (Professional Edition)
Build #PY-233.13135.95, built on December 19, 2023

Runtime version: 17.0.9+7-b1087.9 amd64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
Windows 11.0
GC: G1 Young Generation, G1 Old Generation
Memory: 4054M
Cores: 20
Registry:
ide.experimental.ui=true

[RobertCraigie]

We've added a workaround that will fix this in the next release! I'd still strongly encourage the PyCharm folks to fix this on their end as well.

Note that PyCharm's debugger will attempt to call our module level client as well which can result in an error if the OPENAI_API_KEY environment variable is set, so you may have to ensure that environment variable is set to be able to debug.