Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
WaitIterators: only return a given object once #1290
Depending on feedback, this PR may supersede #1288.
There are detailed commit messages describing what is going on. As a summary, this PR:
Other than documentation, the alternative would be to break all calls to
I did this by decythonizing the WaitIterator class. The pro is that we can now use the Python
One alternative I just thought of would be to split WaitIterator into a cython
Right now a WaitIterator can return the same object many times, which I think is very unexpected and probably pathological. I'm curious for feedback on backward compatibility, but my thought is that the current behavior is so broken (pardon me) that nobody would be relying on it.
PS: I think it might be worthwhile to expose WaitIterator (or something like it) as a documented interface, with
PPS: I had a lot of fun with the test suite while composing these changes. I especially liked the part where it kept track of reference counts and uncollectible garbage for me. Once I got the tests passing (and not a moment before), I was delighted to have such a detailed test suite.
Thanks for the PR! I appreciate the effort put into this.
However, gevent does not use finalizers/
referenced this pull request
Oct 16, 2018
In #1292, you mentioned "Unlinking objects as they become ready breaks the use case that deliberately resets them and lets them become ready, up to count number of times (think of queue processors)." It sounds like you're thinking of a use-case like:
But to make this work would require both a) the user knows the number of iterations they expect, in advance of calling
You've also mentioned that we shouldn't require that objects passed to iwait be hashable. All of the objects which I can think people might use in this context are hashable, including Popen, Semaphore, Event, greenlet and Greenlet. And anyone who makes their own class implementing
I'm thinking back to my postscript in the PR description. If it isn't possible to modify
Oops, the test case should be:
And it fails with:
In a nutshell, those new restrictions are potentially breaking changes, and I don't really see much benefit to them that can't be accomplished today. E.g., if users want to pass in a
One thing I've learned repeatedly maintaining gevent and other projects is that if something is allowed, somebody out there is probably taking advantage of it. Whether or not it's explicitly documented. So making changes that tighten contracts, implicitly or explicitly, is a careful balancing act between a lot of factors. (I freely admit I don't always get that balance right.) (One quick semi-related example: people sometimes pass what are supposed to be positional parameters as keyword arguments, even to internal functions; changing a positional parameter name can wind up breaking code. There's a whole part of the test suite that makes sure that positional parameter names match the standard lib because of that.)
Not if they implement
I think the primitive building blocks for that are already public and that could be done today if it was really needed. To consider extending gevent's interface in that direction I'd need to see real use-cases, ideally in different codebases, to suggest that it's a common enough need that can't be easily met otherwise.
I can imagine that there might be a use-case for optionally unlinking objects as they are yielded, but that seems like it's rare, and if that's a problem it might be that the design is too complicated (i.e., i-waiting on and setting the same event from multiple greenlets at the same time is some sort of complicated multi-fan-in-fan-out thing that can probably be simplified). But I haven't thought too much about it.
Thanks for the detailed explanation. I'd like to clarify two points and suggest one idea.
The problem raised in this PR is not related to the input of
Ooh, I didn't know that. Thanks for informing me. As a workaround (and assuming we wanted to go ahead with any version of this PR), we could potentially use a
Thanks again for your responses. If any of this is news for you, I'd be happy to continue iterating on this PR. If otherwise, it's probably time to close discussion.