New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-14976: Reentrant simple queue #3346

Merged
merged 29 commits into from Jan 15, 2018

Conversation

Projects
None yet
7 participants
@pitrou
Member

pitrou commented Sep 5, 2017

https://bugs.python.org/issue14976

TODO:

  • Add reentrancy test for C version
  • Update Windows build files
  • Add docs

pitrou added some commits Sep 5, 2017

@pitrou

This comment has been minimized.

Show comment
Hide comment
@pitrou

pitrou Sep 5, 2017

Member

Hmm, would someone want to update the Windows build files for the new _queue extension for me? @zooba, @pfmoore, @zware perhaps?

Member

pitrou commented Sep 5, 2017

Hmm, would someone want to update the Windows build files for the new _queue extension for me? @zooba, @pfmoore, @zware perhaps?

@rhettinger

When the docs are made, the reentrancy guarantee should be listed as a C-implementation detail.

Show outdated Hide outdated Lib/queue.py
self = (simplequeueobject *) type->tp_alloc(type, 0);
if (self != NULL) {
self->weakreflist = NULL;
self->lst = PyList_New(0);

This comment has been minimized.

@rhettinger

rhettinger Sep 6, 2017

Collaborator

Can you swap in a deque() using PyObject_Call?

@rhettinger

rhettinger Sep 6, 2017

Collaborator

Can you swap in a deque() using PyObject_Call?

This comment has been minimized.

@pitrou

pitrou Sep 6, 2017

Member

It would replace all the simple PyList C API calls with more cumbersome method calls through the PyObject API. Also, since generic method calls can allocate tuples, they might release the GIL (though I haven't checked for sure). Do you think it's worth it?

@pitrou

pitrou Sep 6, 2017

Member

It would replace all the simple PyList C API calls with more cumbersome method calls through the PyObject API. Also, since generic method calls can allocate tuples, they might release the GIL (though I haven't checked for sure). Do you think it's worth it?

This comment has been minimized.

@rhettinger

rhettinger Sep 7, 2017

Collaborator

Agreed. Go ahead and stick with the PyList calls.

@rhettinger

rhettinger Sep 7, 2017

Collaborator

Agreed. Go ahead and stick with the PyList calls.

@zooba

zooba approved these changes Sep 6, 2017

Approval for the Windows build files - they look fine.

@pitrou pitrou changed the title from [WIP] bpo-14976: Reentrant simple queue to bpo-14976: Reentrant simple queue Sep 7, 2017

@pitrou

This comment has been minimized.

Show comment
Hide comment
@pitrou

pitrou Oct 23, 2017

Member

@rhettinger, what should be the way forward here? Do you have further requests on this PR?

Member

pitrou commented Oct 23, 2017

@rhettinger, what should be the way forward here? Do you have further requests on this PR?

@pitrou

This comment has been minimized.

Show comment
Hide comment
@pitrou

pitrou Dec 19, 2017

Member

@rhettinger could I have your feedback on this?

Member

pitrou commented Dec 19, 2017

@rhettinger could I have your feedback on this?

@pitrou pitrou requested a review from python/windows-team as a code owner Dec 19, 2017

@gpshead

overall good, a couple comments to think about but nothing worth blocking this.

Show outdated Hide outdated Lib/test/test_queue.py
class _PySimpleQueue:
'''Simple, unbounded FIFO queue.
This pure Python implementation is not reentrant.

This comment has been minimized.

@gpshead

gpshead Jan 15, 2018

Member

What is the point of the pure python version existing at all of it's raison d'être isn't possible? Even on PyPy if they took this module and haven't implemented a _queue equivalent it would do the wrong thing for code that wants to use SimpleQueue for reentrant API reasons.

@gpshead

gpshead Jan 15, 2018

Member

What is the point of the pure python version existing at all of it's raison d'être isn't possible? Even on PyPy if they took this module and haven't implemented a _queue equivalent it would do the wrong thing for code that wants to use SimpleQueue for reentrant API reasons.

This comment has been minimized.

@pitrou

pitrou Jan 15, 2018

Member

I know its presence is debattable, but providing a pure Python version is current policy. Besides, being reentrant is documented as an implementation detail, not an intrinsic property of the API.

@pitrou

pitrou Jan 15, 2018

Member

I know its presence is debattable, but providing a pure Python version is current policy. Besides, being reentrant is documented as an implementation detail, not an intrinsic property of the API.

@serhiy-storchaka

The size of the internal list shouldn't be included in the result of __sizeof__.

Show outdated Hide outdated Modules/_queuemodule.c
Show outdated Hide outdated Modules/_queuemodule.c
0, /*tp_iter*/
0, /*tp_iternext*/
simplequeue_methods, /*tp_methods*/
0, /* tp_members */

This comment has been minimized.

@serhiy-storchaka

serhiy-storchaka Jan 15, 2018

Member

Inconsistent use of spaces in comments.

@serhiy-storchaka

serhiy-storchaka Jan 15, 2018

Member

Inconsistent use of spaces in comments.

This comment has been minimized.

@pitrou

pitrou Jan 15, 2018

Member

I will happily let others reformat if they're irritated by inconsistent spaces in comments.

@pitrou

pitrou Jan 15, 2018

Member

I will happily let others reformat if they're irritated by inconsistent spaces in comments.

Show outdated Hide outdated Modules/_queuemodule.c
Show outdated Hide outdated Modules/_queuemodule.c
@bedevere-bot

This comment has been minimized.

Show comment
Hide comment
@bedevere-bot

bedevere-bot Jan 15, 2018

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

bedevere-bot commented Jan 15, 2018

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

@pitrou pitrou merged commit 94e1696 into python:master Jan 15, 2018

4 checks passed

bedevere/issue-number Issue number 14976 found
Details
bedevere/news News entry found in Misc/NEWS.d
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@pitrou pitrou deleted the pitrou:simple_queue branch Jan 15, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment