New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
need patch queue.SimpleQueue for py37 #1248
Comments
Another interesting question I want to know is why not patch the whole queue module by |
There are native thread use cases for the original objects. |
In my test, combine use the native thread and the monkey patched standard library sometimes will cause deadlock but I can't confirm when and where cause that. So I rewrite that code by use |
py3.7 deadlock with monkey patch of stdlib thread modules + use of ThreadPoolExecutor 3.7, ThreadPoolExecuter was changed to use queue.SimpleQueue; on 3.6 it uses queue.Queue. No issue in 3.6 The python37 default SimpleQueue was implemented with C code, which will cause hang the main thread. Commit 2a71b8e introduce eventlet for unittest, but I did not find why we need it. Check other project they also use ThreadPoolExecutor, but not all of them introduce eventlet. Maybe GreenThreadPoolExecutor is another solution. Ref: ThreadPoolExecutor https://docs.python.org/3.7/library/concurrent.futures.html#threadpoolexecutor Same isses about ThreadPoolExecutor and eventlet https://bugs.launchpad.net/designate/+bug/1782647 eventlet/eventlet#508 gevent/gevent#1253 ClericPy/torequests#10 https://bugs.python.org/issue34173 gevent/gevent#1248 gevent/gevent#1251 http://lists.openstack.org/pipermail/openstack-dev/2018-July/132473.html https://www.gitmemory.com/issue/eventlet/eventlet/508/511101004 Change-Id: I263f7222119d8d0c89cfc2a4758fe376ce1afd60
* Update cyborg from branch 'master' - Merge "change default SimpleQueue to _PySimpleQueue for queue" - change default SimpleQueue to _PySimpleQueue for queue py3.7 deadlock with monkey patch of stdlib thread modules + use of ThreadPoolExecutor 3.7, ThreadPoolExecuter was changed to use queue.SimpleQueue; on 3.6 it uses queue.Queue. No issue in 3.6 The python37 default SimpleQueue was implemented with C code, which will cause hang the main thread. Commit 2a71b8e introduce eventlet for unittest, but I did not find why we need it. Check other project they also use ThreadPoolExecutor, but not all of them introduce eventlet. Maybe GreenThreadPoolExecutor is another solution. Ref: ThreadPoolExecutor https://docs.python.org/3.7/library/concurrent.futures.html#threadpoolexecutor Same isses about ThreadPoolExecutor and eventlet https://bugs.launchpad.net/designate/+bug/1782647 eventlet/eventlet#508 gevent/gevent#1253 ClericPy/torequests#10 https://bugs.python.org/issue34173 gevent/gevent#1248 gevent/gevent#1251 http://lists.openstack.org/pipermail/openstack-dev/2018-July/132473.html https://www.gitmemory.com/issue/eventlet/eventlet/508/511101004 Change-Id: I263f7222119d8d0c89cfc2a4758fe376ce1afd60
Description:
The python37 default
SimpleQueue
was implemented with C code, which will cause hang the main thread.A simple way to solve this problem like that
but I think this code is so ugly and may be have better way to implements in gevent.
The text was updated successfully, but these errors were encountered: