Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

0.29.1: RecursionError: maximum recursion depth exceeded while calling a Python object #677

Closed
yuezhongtao opened this issue Dec 11, 2020 · 3 comments

Comments

@yuezhongtao
Copy link

@yuezhongtao yuezhongtao commented Dec 11, 2020

eventlet version:
Name : python3-eventlet
Version : 0.29.1
Release : 412

os:
Linux E2E0A942-2AB1-8546-AB44-B994FF1EB2B2 4.18.0-147.5.1.0.h276.eulerosv2r9.x86_64 #1 SMP Thu Nov 26 17:26:42 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

python: Python 3.7.4

crash error trace log:

2020-12-11T14:56:40.665+08:00 localhost neutron-server ERROR [pid:363233] [GreenThread-1835][tid:22489070077920] [catch_errors.py:44 __call__] An error occurred during processing the request: GET /v2.0/networks?name=external_om HTTP/1.0^M
Accept: application/json^M
Accept-Encoding: gzip, deflate^M
Connection: close^M
Content-Type: application/json^M
Host: network.localdomain.com:8020^M
User-Agent: python-requests/2.22.0^M
X-Auth-Token: *****: RecursionError: maximum recursion depth exceeded while calling a Python object
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors Traceback (most recent call last):
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/oslo_middleware/catch_errors.py", line 40, in __call__
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     response = req.get_response(self.application)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/webob/request.py", line 1314, in send
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     application, catch_exc_info=False)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/webob/request.py", line 1278, in call_application
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     app_iter = application(self.environ, start_response)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/webob/dec.py", line 129, in __call__
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     resp = self.call_func(req, *args, **kw)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/webob/dec.py", line 193, in call_func
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     return self.func(req, *args, **kwargs)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/osprofiler/web.py", line 122, in __call__
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     return request.get_response(self.application)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/webob/request.py", line 1314, in send
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     application, catch_exc_info=False)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/webob/request.py", line 1278, in call_application
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     app_iter = application(self.environ, start_response)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/webob/dec.py", line 129, in __call__
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     resp = self.call_func(req, *args, **kw)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/webob/dec.py", line 193, in call_func
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     return self.func(req, *args, **kwargs)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 338, in __call__
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     response = self.process_request(req)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 660, in process_request
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     resp = super(AuthProtocol, self).process_request(request)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 411, in process_request
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     allow_expired=allow_expired)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 446, in _do_fetch_token
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     data = self.fetch_token(token, **kwargs)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 755, in fetch_token
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     allow_expired=allow_expired)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystonemiddleware/auth_token/_identity.py", line 180, in verify_token
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     allow_expired=allow_expired)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystonemiddleware/auth_token/_identity.py", line 75, in verify_token
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     access_rules_support=ACCESS_RULES_SUPPORT)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneclient/v3/tokens.py", line 120, in validate
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     access_rules_support=access_rules_support)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneclient/v3/tokens.py", line 93, in get_token_data
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     resp, body = self._client.get(url, headers=headers)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneclient/v3/tokens.py", line 93, in get_token_data
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     resp, body = self._client.get(url, headers=headers)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/adapter.py", line 386, in get
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     return self.request(url, 'GET', **kwargs)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/adapter.py", line 545, in request
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     resp = super(LegacyJsonAdapter, self).request(*args, **kwargs)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/adapter.py", line 248, in request
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     return self.session.request(url, method, **kwargs)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/adapter.py", line 248, in request
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     return self.session.request(url, method, **kwargs)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/session.py", line 271, in inner
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     return func(*args, **kwargs)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/session.py", line 766, in request
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     auth_headers = self.get_auth_headers(auth)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/session.py", line 1177, in get_auth_headers
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     return auth.get_headers(self, **kwargs)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/plugin.py", line 95, in get_headers
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     token = self.get_token(session)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/identity/base.py", line 88, in get_token
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     return self.get_access(session).auth_token
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/identity/base.py", line 134, in get_access
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     self.auth_ref = self.get_auth_ref(session)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/identity/generic/base.py", line 207, in get_auth_ref
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     self._plugin = self._do_create_plugin(session)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/identity/generic/base.py", line 138, in _do_create_plugin
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     authenticated=False)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/identity/base.py", line 610, in get_discovery
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     authenticated=authenticated)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/discover.py", line 1442, in get_discovery
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     disc = Discover(session, url, authenticated=authenticated)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/discover.py", line 526, in __init__
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     authenticated=authenticated)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/discover.py", line 101, in get_version_data
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     resp = session.get(url, headers=headers, authenticated=authenticated)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/session.py", line 1117, in get
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     return self.request(url, 'GET', **kwargs)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/session.py", line 271, in inner
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     return func(*args, **kwargs)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/session.py", line 907, in request
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     resp = send(**kwargs)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/keystoneauth1/session.py", line 998, in _send_request
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     resp = self.session.request(method, url, **kwargs)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/requests/sessions.py", line 533, in request
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     resp = self.send(prep, **send_kwargs)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/requests/sessions.py", line 646, in send
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     r = adapter.send(request, **kwargs)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     timeout=timeout
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/urllib3/connectionpool.py", line 667, in urlopen
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/urllib3/connectionpool.py", line 945, in _prepare_proxy
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/urllib3/connection.py", line 345, in connect
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     default_ssl_context = False
2020-12-11 14:56:20.591 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib64/python3.7/ssl.py", line 518, in options
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/urllib3/connection.py", line 345, in connect
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     default_ssl_context = False
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib/python3.7/site-packages/urllib3/util/ssl_.py", line 277, in create_urllib3_context
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     # (issue #309)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib64/python3.7/ssl.py", line 518, in options
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     super(SSLContext, SSLContext).options.__set__(self, value)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib64/python3.7/ssl.py", line 518, in options
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     super(SSLContext, SSLContext).options.__set__(self, value)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   File "/usr/lib64/python3.7/ssl.py", line 518, in options
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors     super(SSLContext, SSLContext).options.__set__(self, value)
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors   [Previous line repeated 423 more times]
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors RecursionError: maximum recursion depth exceeded while calling a Python object
2020-12-11 14:56:40.660 363233 ERROR oslo_middleware.catch_errors

@temoto
Copy link
Member

@temoto temoto commented Dec 11, 2020

Probably duplicate #371 #618

Please post minimal reproduction code.

Loading

@thomasgoirand
Copy link

@thomasgoirand thomasgoirand commented Jan 28, 2021

Hi Temoto. No, this isn't a duplicate, but a new issue. I'm experiencing this with Python 3.9 under Debian Sid as well, with Eventlet 0.26.1 (with some added patches cherry-picked to fix other issues).

Loading

@thomasgoirand
Copy link

@thomasgoirand thomasgoirand commented Feb 2, 2021

Hi. I just understood that the issue is triggered with dnspython 2.0.0 (at least for me). Discussing the issue on the relevant already opened bug from now on.

Loading

tipabu added a commit to tipabu/eventlet that referenced this issue Feb 4, 2021
Newer (2.0+) dnspython imports ssl (and requests, which would have
eventually imported ssl), so greendns needs a monkey-patched ssl, too.
This should prevent a RecursionError with SSLContext.

Note that this *does not* bring full dnspython>=2.0 support, but our
version pin hasn't stopped people from trying to run with newer
dnspython anyway. May as well make it hurt a little less.

Fixes eventlet#677, but see also eventlet#619.
@temoto temoto closed this in #684 Mar 3, 2021
temoto added a commit that referenced this issue Mar 3, 2021
Newer (2.0+) dnspython imports ssl (and requests, which would have
eventually imported ssl), so greendns needs a monkey-patched ssl, too.
This should prevent a RecursionError with SSLContext.

Note that this *does not* bring full dnspython>=2.0 support, but our
version pin hasn't stopped people from trying to run with newer
dnspython anyway. May as well make it hurt a little less.

Fixes #677, but see also #619.
bors bot added a commit to duckinator/parts.horse that referenced this issue Mar 3, 2021
76: Update eventlet to 0.30.2 r=duckinator a=pyup-bot


This PR updates [eventlet](https://pypi.org/project/eventlet) from **0.30.1** to **0.30.2**.



<details>
  <summary>Changelog</summary>
  
  
   ### 0.30.2
   ```
   ======
* greendns: patch ssl to fix RecursionError on SSLContext.options.__set__ eventlet/eventlet#677
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/eventlet
  - Changelog: https://pyup.io/changelogs/eventlet/
  - Homepage: http://eventlet.net
</details>



Co-authored-by: pyup-bot <github-bot@pyup.io>
reimannf added a commit to sapcc/helm-charts that referenced this issue May 28, 2021
Seems that the Sentry logger causes a import of requests before
the monkey patch of the eventlet happens.
eventlet/eventlet#371 (comment)

With Wallaby this seems now be an issue and causes a loop when importing
the modules like here: eventlet/eventlet#677
although it is not dnspython >= 2 related.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

3 participants