New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Egg race condition #1418

Merged
merged 3 commits into from Jul 20, 2018

Conversation

Projects
None yet
2 participants
@pganssle
Member

pganssle commented Jul 11, 2018

Summary of changes

This PR is an approach to #1412 that adds a test to induce this race condition. The test relies on implementation details, but that is made explicit in the text of the test, I have done it this way because there isn't another great way to deterministically induce the problem.

cc: @JonKohler

Closes #1256

Pull Request Checklist

  • Changes have tests
  • News fragment added in changelog.d. See documentation for details

@pganssle pganssle force-pushed the pganssle:egg_race_condition branch from 16ab345 to 07eb87f Jul 11, 2018

@pganssle pganssle referenced this pull request Jul 11, 2018

Closed

Solve race during creation of egg cache (fixes #1256) #1412

2 of 2 tasks complete
@JonKohler

This comment has been minimized.

Contributor

JonKohler commented Jul 11, 2018

Thanks for tackling this, I appreciate it! +1 from me

@pganssle pganssle force-pushed the pganssle:egg_race_condition branch from 07eb87f to e36c32d Jul 11, 2018

pganssle and others added some commits Jul 11, 2018

Add test for cache_path race condition
This mocks out the isdir call so that the directory is created
immediately after you determine whether or not it exists, thus
simulating a race condition between two threads or processes
creating the same directory.
Fix race condition in _bypass_ensure_directory
This fixes a race condition in _bypass_ensure_directory where
two threads or processes may erroneously fail because they are
both creating the same directory. A more robust implementation
of this may involve exposing the un-wrapped os.makedirs.

Originally reported with proposed patch by @JonKohler in
github PR #1412. This patch came out of discussions on that
thread.

@pganssle pganssle force-pushed the pganssle:egg_race_condition branch from 601217f to 67f190a Jul 12, 2018

@JonKohler

This comment has been minimized.

Contributor

JonKohler commented Jul 18, 2018

Where are we at on this? is there anything remaining?

Does this replace my original PR?

@pganssle

This comment has been minimized.

Member

pganssle commented Jul 20, 2018

Yes this should replace your original PR since it solves the same problem in a different way. I've seen no objections since I opened this PR so I'm going to merge it.

@pganssle pganssle merged commit e2dde5b into pypa:master Jul 20, 2018

5 checks passed

codecov/patch 100% of diff hit (target 81.11%)
Details
codecov/project 81.16% (+0.05%) compared to 89155ab
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
deploy/netlify Deploy preview ready!
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment