Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Massively parallel directory creation fail #50

Open
cfrontin opened this issue Apr 29, 2024 · 0 comments
Open

Massively parallel directory creation fail #50

cfrontin opened this issue Apr 29, 2024 · 0 comments

Comments

@cfrontin
Copy link

When running in a massively parallel way on HPC, for instance, this line can fail. It appears that there's a race condition that can get multiple processors past the L320 if statement before this line is reached, then the "if directory exists" condition changes after evaluating the if statement on L320.

Adding a keyword argument for exists_ok=True into the os.makedirs call on L320 can be used to allow this condition to be ok, I think.

@cfrontin cfrontin changed the title Massively parallel fail Massively parallel directory creation fail Apr 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant