Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Import Race Condition #108
If you have two greenlets that import the same module, for example 'foo', and foo.py does something that blocks, the import will fail in one of the greenlets with 'ImportError: cannot import name foo'. This problem only occurs under gevent.
In CPython, there is some C code that handles locking around imports, ensuring that only one import is happening concurrently. This works fine for threads but does not handle greenlet switches. As far as the import lock is concerned, the two greenlets are the same thread, so it happily lets both imports go ahead, resulting in the error seen above.
I tried fixing this by monkey patching
What steps will reproduce the problem?
What is the expected output? What do you see instead?
What version of the gevent are you using?
On what operating system?
On what Python?
Both gevent and threading scripts fail for me:
Should test_thread.py in the attached tar.gz run without exceptions?
firstname.lastname@example.org said, at 2012-01-03T22:57:55.000Z:
When I was cleaning up the example before submitting, I removed the "x = 'foo'" line from foo.py, causing the "from foo import x" line to fail for obvious reasons. Bad form on my part for not re-checking it after "cleanup". Sorry about that -- here's a tarball with the correct foo.py and a patch that monkey patches
Denis.Bilenko said, at 2012-01-06T10:55:34.000Z:
email@example.com said, at 2012-01-07T22:18:55.000Z:
None that I've seen thus far. We've been running a version of gevent with this patch in place for a couple of weeks without any issues. I'm not particularly happy about the check for string and unicode types, but I couldn't think of a cleaner way to handle the issue. If someone is using a custom importer and bypassing
If there's a better way to monkey-patch builtin functions that handles the bound method case properly, we should use that instead, but my searches turned up nothing thus far.
As for the using the RLock around the import, that should be perfectly safe. CPython does something equivalent to protect the threaded case -- this effectively just extends that protection to greenlets as well.
schmir said, at 2012-01-17T22:17:28.000Z:
FYI, we had to use an ugly workaround for that issue.