Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Dask temporary file utility #5361

Merged
merged 5 commits into from
Sep 29, 2021
Merged

Conversation

jrbourbeau
Copy link
Member

@jrbourbeau jrbourbeau commented Sep 27, 2021

This PR deprecates distributed.utils.tmpfile in favor of using the upstream dask.utils.tmpfile utility

Supersedes #4912

cc @ncclementi

@ncclementi
Copy link
Member

It seems we have legit failings on Windows, we are getting a

PermissionError: [WinError 32]  The process cannot access the file because it is being used by another process ...

Tried to look into but couldn't figure out what is actually causing the problem.:(

@jrbourbeau
Copy link
Member Author

It turns out the tmpfile utility in distributed caught an OSError in one more place than the tmpfile utility in dask did. I'm rerunning tests here against this branch https://github.com/jrbourbeau/dask/tree/tmpfile-oserror which updates the dask tmpfile utility to handle the extra OSError edge case. If this fixes tests here, then I'll push a PR up to dask proposing we modify tmpfile there

@jrbourbeau
Copy link
Member Author

Alright, that seems to fix things. Here's the corresponding dask PR dask/dask#8191. We can revert the CI changes here and merge this PR after dask/dask#8191 has been merged.

@jrbourbeau
Copy link
Member Author

Thanks for reviewing @ncclementi!

@jrbourbeau jrbourbeau merged commit 43d3866 into dask:main Sep 29, 2021
@jrbourbeau jrbourbeau deleted the dask-tmpfile branch September 29, 2021 20:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants