Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
In all versions of CPython, sets are iterated according to increasing hash order. When a Semaphore is released, it notifies only one waiter. That waiter happens to be the first waiter that's in a set of waiters. This means that if a new waiter is added to a semaphore while an old waiter is already there, but the new waiter has a lower hashcode, it will take priority over the old waiter. (The waiters here are
To put that in concrete terms, have one greenlet acquire a semaphore. Have a second greenlet attempt to acquire the semaphore and block. Have the first greenlet spawn a new greenlet that wants to acquire the semaphore (and repeat the process). If the hash codes work out just so, that newly spawned greenlet will get the semaphore, while the original waiting greenlet continues to wait indefinitely.
In this example, we contrive to manipulate the hash codes of
from __future__ import print_function import sys import gevent from gevent import getcurrent from gevent.lock import Semaphore class Switch: def __init__(self, greenlet, hashcode): self.switch = greenlet.switch self.hashcode = hashcode def __hash__(self): return self.hashcode def __eq__(self, other): return self is other def __call__(self, *args, **kwargs): return self.switch(*args, **kwargs) class FirstG(gevent.Greenlet): hashcode = 10 def __init__(self, *args, **kwargs): gevent.Greenlet.__init__(self, *args, **kwargs) self.switch = Switch(self, self.hashcode) class LastG(FirstG): hashcode = 12 sem = Semaphore() def acquire_then_exit(): print("In", getcurrent(), "About to acquire for exit") sem.acquire() print("In", getcurrent(), "Acquired and exiting") sys.exit() def acquire_then_spawn(): print("In", getcurrent(), "About to acquire to spawn") sem.acquire() print("In", getcurrent(), "acquired; about to spawn") g = FirstG.spawn(release_then_spawn) print("In", getcurrent(), "spawned", g, "; about to join") g.join() print("In", getcurrent(), "joined and done") def release_then_spawn(): print("in", getcurrent(), "about to release") sem.release() print("in", getcurrent(), "spawning") g = FirstG.spawn(acquire_then_spawn) print("in", getcurrent(), "joining") g.join() keep_going1 = FirstG.spawn(acquire_then_spawn) keep_going2 = FirstG.spawn(acquire_then_spawn) exiting = LastG.spawn(acquire_then_exit) gevent.joinall([keep_going1, keep_going2, exiting])
Note: I wasn't able to get the non-termination to happen without manipulating the hashcodes. I think this is because usually later allocated method objects will have higher addresses than earlier ones, and the address is the hashcode. But this all depends on the internals of Python memory allocation. Setting
This applies to things built on top of
Now, the Python documentation says those lock types aren't necessarily fair. On Windows, they use semaphores, which aren't guaranteed to be fair AFAICS. On Unix systems, they're implemented with
Fairness is a desired attribute if it's cheap. Fairness doesn't necessarily have to mean FIFO, but if that's cheap then that would also be nice.