-
Notifications
You must be signed in to change notification settings - Fork 105
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
make_numbered_dir() multi-process-safe #30
Comments
Original comment by @arigo Ah, good solution. +1 |
Original comment by @RonnyPfannschmidt the problem i see is the combination of 2 different issues with just that patch\ the solution to just the race condition is the following
the issue with keeping things alive time based is another one |
@RonnyPfannschmidt: two issues with mkdir + rename strategy:
@arigo: leaving tmpdirs for 5 minutes by default could make tmp size unacceptable, if test sessions are short and generate lot of filesystem data. Flock strategy would be best, but has limitations (not all filesystems, not all OS) and demands external library or 500+ lines of code. My proposition is this:
Race problems with different implementations can be reproduced using this tiny suite: |
closing as solved by #143 |
py.path.local.make_numbered_dir() has a race condition that only shows up if more than "keep" processes start up at exactly the same time (which could occur when starting tests in parallel).
The fix I provide here, while not theoretically perfect, should be good enough. It prevents very recent directories from being deleted (even if they don't have (yet?) a .lock file in them). For what "very recent" means, I suppose that any value greater than 1 second is good enough, but I personally like to have enough time to get a chance to copy a directory that I want to keep, so the patch uses 5 minutes as the default.
https://bitbucket.org/pypy/pypy/commits/c6f52c21fe7e0e5385764cb5b1013cfbc39884be
The text was updated successfully, but these errors were encountered: