Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Be resilient against parallel directory creations. #75

Merged
merged 4 commits into from
Jul 9, 2018

Conversation

xhochy
Copy link
Contributor

@xhochy xhochy commented Jun 12, 2018

When using multiprocessing, simultaneous writes can lead to the
situation where both code paths first check if the directory exists and
create it in the negative case. As check and creation aren't a single
atomic operation, one of them might fail.

@xhochy
Copy link
Contributor Author

xhochy commented Jun 12, 2018

Failures in Travis are unrelated (minio does not start)

@criemen
Copy link
Collaborator

criemen commented Jun 13, 2018

Fixed the problem with minio on master.

@xhochy xhochy force-pushed the ensure-thread-safety-makedirs branch from fe5d0c2 to 54f7e05 Compare June 14, 2018 12:20
@coveralls
Copy link

coveralls commented Jun 14, 2018

Coverage Status

Coverage decreased (-8.4%) to 81.182% when pulling 3d9f53d on xhochy:ensure-thread-safety-makedirs into 599788b on mbr:master.

@xhochy
Copy link
Contributor Author

xhochy commented Jun 14, 2018

Rebased and Travis is happy. The decrease in coverage seems to be down to the Azure tests being skipped, this should be irrelevant for this branch.

@fmarczin fmarczin self-requested a review June 21, 2018 09:37
@xhochy xhochy force-pushed the ensure-thread-safety-makedirs branch from a8d2584 to 098daec Compare June 22, 2018 12:56
Copy link
Collaborator

@criemen criemen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a changelog entry, else it looks good 👍

When using multiprocessing, simultaneous writes can lead to the
situation where both code paths first check if the directory exists and
create it in the negative case. As check and creation aren't a single
atomic operation, one of them might fail.
@xhochy xhochy force-pushed the ensure-thread-safety-makedirs branch from 839292d to 3d9f53d Compare July 5, 2018 07:53
@xhochy
Copy link
Contributor Author

xhochy commented Jul 5, 2018

Rebased and added changelog entry.

@fmarczin fmarczin merged commit 9ac9f3a into mbr:master Jul 9, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants