Windows: Workaround EACCES during rename() #1363
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
TODO: Add retry limit |
Merged
ManasJayanth
added a commit
that referenced
this pull request
Jan 6, 2022
Retry rename on EACCES This PR retries `rename` upon getting `EACCES`. I've included data about how many retries are likely necessary. On my system (WSL1 Ubuntu 20.04, omitting hardware details), the `EACCES` issue makes it impossible to use esy to install any of the Dream examples or use Dream's quick start. As the data below shows, every installation is expected to fail with `EACCES`, if it is not worked around. The underlying `EACCES` issue seems to be a long-standing problem on WSL1 (microsoft/WSL#1529, microsoft/WSL#3395), and I think we do have to work around it in esy. I'm not sure what is causing the `EACCES` exactly. I think there are two main classes of possibilities: - Self-interaction between esy's opened file descriptors and `rename`. I think the self-interaction is due to WSL rather than Lwt or another library. Since I compiled esy on WSL, it is using Lwt's Unix (rather than Windows) C code. Since the Unix code seems to work fine on Linux and Mac, this suggests a WSL issue. - Interaction between esy and file indexers or other proceses running on the system. I'm not sure if that's a WSL issue or not, but I've never had to be aware of such processes when doing renames in Cygwin or elsewhere. I built esy with this patch under WSL and ran clean `esy install`s in Dream's [full-stack ReScript](https://github.com/aantron/dream/tree/03e4d37cb5f5f638707479cd46105e2ee2b1df0e/example/w-fullstack-rescript#readme) example, using this script: ```sh #!/bin/bash export PATH="/home/antron/code/attic/esy/_build/install/default/bin:$PATH" export ESY__PREFIX="/home/antron/code/dream/dream/example/w-fullstack-rescript/esy-prefix" export OCAMLRUNPARAM=b RUN=1 while true do rm -rf esy-prefix _esy esy.lock lib node_modules/ package-lock.json echo echo "RUN $RUN" which esy esy install # --verbosity debug if [ $? != 0 ] then exit fi RUN=$((RUN+1)) done ``` The example was checked out into NTFS. The system was freshly restarted, and VSCode (or anything similar) was not running. I used a version of this patch with a print showing the number of attempts before `rename` succeeds, and got the following results from 5 runs: ``` 1 attempt: 802 2 attempts: 52 3 attempts: 12 4 attempts: 3 5 attempts: 1 total: 870 ``` Based on this, I naively estimated that if a `rename` needs more than 1 attempt, the number of attempts needed decays by a factor of 4 at each step. I set the limit on the number of attempts naively to 8, thus expecting one failure in about 500 `esy install` attempts of the Dream ReScript example, under all these simplified assumptions. The delay between attempts is (over) one second, so this means that upon legitimate `EACCES`, users will have to wait eight seconds to get an error message. I think there are two ways to address this: - Fall back to recursive copy rather than retrying `rename` when `rename` fails. Do we have a recursive copy available in esy or its dependencies? Is it fine to leave the source directory intact? - Detect WSL and retry only on WSL. Waiting for 8 seconds is still a much better user experience than failure to install at all, so the PR will still be an improvement, without, in this case, harming Linux or Mac users. We could also add a message, shown in case we finally fail with `EACCES` on WSL, giving users a hint about potential VSCode or other watchers, and what else they can try to solve the problem. Closes #1363. Probably fixes #1097, some of the reports after the first one. Probably fixes #1083. Probably fixes #593, but I haven't looked into non-WSL Windows yet. Probably fixes aantron/dream#63. cc @bryphe, @rizo, @jordwalke, @iMplode-nZ, @a-c-sreedhar-reddy, @srirajshukla, @andreypopp
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR retries
rename()if it encounters anEACCESS