Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

monkey.patch_socket(aggressive=True) no longer removes select.epoll (1.5 regression) #1570

Closed
navytux opened this issue Apr 16, 2020 · 6 comments · Fixed by #1571
Closed

monkey.patch_socket(aggressive=True) no longer removes select.epoll (1.5 regression) #1570

navytux opened this issue Apr 16, 2020 · 6 comments · Fixed by #1571

Comments

@navytux
Copy link

@navytux navytux commented Apr 16, 2020

  • gevent version: gevent-1.5.0-cp37-cp37m-manylinux2010_x86_64.whl
  • Python version: CPython 3.7.3 (default, Dec 20 2019, 18:57:59) provided by Debian10
  • Operating System: Debian GNU/Linux 10, Linux deco 4.19.0-8-amd64 #1 SMP Debian 4.19.98-1 (2020-01-26) x86_64 GNU/Linux

Description:

monkey.patch_socket(aggressive=True) documents that it removes things like select.epoll on Py3. It used to be indeed so with gevent-1.4, but is no longer the case with gevent-1.5:

(3g14.venv) kirr@deco:~/tmp/trashme/r$ python
Python 3.7.3 (default, Dec 20 2019, 18:57:59) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import gevent
>>> gevent.version_info
version_info(major=1, minor=4, micro=0, releaselevel='dev', serial=0)
>>> from gevent.monkey import patch_select
>>> patch_select(aggressive=True)
>>> import select
>>> select.epoll
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'select' has no attribute 'epoll'       #  <-- NOTE removed
(3.venv) kirr@deco:~/tmp/trashme/r$ python
Python 3.7.3 (default, Dec 20 2019, 18:57:59) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import gevent
>>> gevent.version_info
version_info(major=1, minor=5, micro=0, releaselevel='dev', serial=0)
>>> from gevent.monkey import patch_select
>>> patch_select(aggressive=True)
>>> import select
>>> select.epoll
<class 'select.epoll'>                                         # <-- NOTE not removed

gevent/select.py says:

    if aggressive:
        # since these are blocking we're removing them here. This makes some other
        # modules (e.g. asyncore)  non-blocking, as they use select that we provide
        # when none of these are available.
        patch_request.remove_item(
            'epoll'
            'kqueue',
            'kevent',
            'devpoll',
        )

so judging by this comment, since select.epoll is no longer removed, something might have become broken with e.g. asyncore.

The issue was found via gpython test.

jamadden added a commit that referenced this issue Apr 16, 2020
A missing comma led to string concatenation and a silent bug. I'm surprised this wasn't really tested anywhere.

Fixes #1570
@navytux
Copy link
Author

@navytux navytux commented Apr 16, 2020

@jamadden thanks for quick fix.

@jamadden
Copy link
Member

@jamadden jamadden commented Apr 22, 2020

This was also a problem for kqueue/kevent:

Traceback (most recent call last):
 File "//2.7/lib/python2.7/threading.py", line 801, in __bootstrap_inner
   self.run()
 File "//2.7/lib/python2.7/threading.py", line 754, in run
   self.__target(*self.__args, **self.__kwargs)
 File "///eggs/ZEO-5.2.1-py2.7.egg/ZEO/asyncio/client.py", line 858, in run
   loop = new_event_loop()
 File "///eggs/trollius-2.2-py2.7.egg/trollius/events.py", line 615, in new_event_loop
   return get_event_loop_policy().new_event_loop()
 File "///eggs/trollius-2.2-py2.7.egg/trollius/events.py", line 566, in new_event_loop
   return self._loop_factory()
 File "///eggs/trollius-2.2-py2.7.egg/trollius/unix_events.py", line 56, in __init__
   super(_UnixSelectorEventLoop, self).__init__(selector)
 File "///eggs/trollius-2.2-py2.7.egg/trollius/selector_events.py", line 75, in __init__
   self._make_self_pipe()
 File "///eggs/trollius-2.2-py2.7.egg/trollius/selector_events.py", line 140, in _make_self_pipe
   self.add_reader(self._ssock.fileno(), self._read_from_self)
 File "///eggs/trollius-2.2-py2.7.egg/trollius/selector_events.py", line 257, in add_reader
   (handle, None))
 File "///eggs/trollius-2.2-py2.7.egg/trollius/selectors.py", line 530, in register
   kev = select.kevent(key.fd, select.KQ_FILTER_READ,
AttributeError: 'module' object has no attribute 'kevent'

@navytux
Copy link
Author

@navytux navytux commented Apr 22, 2020

@jamadden, thanks for heads up. Given that this regression is showing in many places, in my view it makes sense to make a 1.5.post1 gevent release with the fix.

@navytux
Copy link
Author

@navytux navytux commented Apr 22, 2020

Sorry, I was too fast to type - you've already made it. Thanks!

@jamadden
Copy link
Member

@jamadden jamadden commented Apr 22, 2020

post releases are reserved for packaging fixes that don't affect the code. The next bugfix release, using the new CalVer scheme, is now out (though I haven't finished updating all the docs).

@navytux
Copy link
Author

@navytux navytux commented Apr 22, 2020

I see; thanks for clarifying.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants