Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EACCES when renaming folder that is being watched from nodejs #3395

Closed
alexdima opened this issue Jul 20, 2018 · 10 comments
Closed

EACCES when renaming folder that is being watched from nodejs #3395

alexdima opened this issue Jul 20, 2018 · 10 comments

Comments

@alexdima
Copy link
Member

alexdima commented Jul 20, 2018

Microsoft Windows [Version 10.0.17134.112]

I am trying to use chokidar, a popular nodejs file system watcher library.

Whenever I use chokidar to listen for file system changes, I can no longer rename folders that lie within the folder being watched.

  • npm install chokidar
  • mkdir test
  • echo hello > test/test.txt
  • create repro.js:
var fs = require('fs');
var chokidar = require('chokidar');

// Watch current directory and ignore the node_modules folder
chokidar.watch('.', {ignored: /node_modules/}).on('all', (event, path) => {
  console.log(event, path);
});

// After sufficient time, attempt to rename the test folder
// Note: this is also reproducible using `mv` on the command line
setTimeout(function() {
    fs.rename('test', 'test2', function(err) {
        if (err) {
            console.log(err);
        } else {
            console.log('rename ok');
        }
    });
}, 1000);

WSL (Ubuntu 18.04):
image

Ubuntu 18.04:
image

I don't think it is terribly important (other colleagues are able to reproduce with other node versions), but I am using nodejs v8.11.3

@therealkenc
Copy link
Collaborator

I don't think it is terribly important (other colleagues are able to reproduce with other node versions)

Nah it's a manifestation of about the most pervasive bug around here (if you don't count 0xhexdigits fail at startup). Call this dupe #1956 but you can pick any dating back to #14.

@qwabra
Copy link

qwabra commented May 20, 2020

this does not work for me

code.visualstudio.com/docs

// settings.json
{
    "remote.WSL.fileWatcher.polling": true
}

but this is helped

// settings.json
{
    "files.watcherExclude": {
        "**/**/*": true
    }
}

@95th
Copy link

95th commented May 21, 2020

@qwabra I tried your approach and even that doesn't work.

I tried:

    "remote.WSL.fileWatcher.polling": true,

then

    "files.watcherExclude": {
        "**/**/*": true
    }

then both.

None of this works for me.

Can anyone please help?

@JordanSimba
Copy link

This is pretty annoying... for now I just close VS code since the file watcher has a lock on the files, do file work in terminal or file explorer and reopen when done with code . in ubuntu.
Otherwise, unless there is a reason to not upgrade to WSL 2 I would suggest anybody does so.

@DerekNonGeneric
Copy link

Why is the workaround from #3395 (comment) not explicitly stated in the VS Code docs section about known WSL limitations? It should note that all workspace files must be excluded from being watched otherwise the problem will persist…

@qwabra
Copy link

qwabra commented Jul 13, 2020

...
for now I just close VS code since the file watcher has a lock on the files, do file work in terminal or file explorer and reopen
...

you can just open settings.json and switch trigger false/true in "**/*": false

{
    "files.watcherExclude": {
        //
        // block Svelte update
        "**/*": false,
        // "**/*": true,
        //
        "**/.git/objects/**": true,
        "**/.git/subtree-cache/**": true,
        "**/node_modules/**": true,
        "**/.hg/store/**": true
    },
}

it can be set locally

(use workspace settings) .vscode/settings.json
see docs

@Spongman
Copy link

something is fundamentally broken here. while having my remote-wsl workspace open in vscode, i get the following in a command prompt:

C:\WINDOWS\system32>handle my-directory-name | wsl wc -l
177481

that means the windows kernel has every single one of the files in my workspace open.

i have the following in my settings.json:

	"remote.WSL.fileWatcher.polling": true

@byteroll
Copy link

this does not work for me

code.visualstudio.com/docs

// settings.json
{
    "remote.WSL.fileWatcher.polling": true
}

but this is helped

// settings.json
{
    "files.watcherExclude": {
        "**/**/*": true
    }
}

Thank you. This also solved my problem in installing go tools on WSL.

@SanjivG10
Copy link

// settings.json (located in %appdata%/code/user/settings.json)
{
"files.watcherExclude": {
"//*": true
}
}

After saving it, restart VsCode.

ManasJayanth added a commit to esy/esy that referenced this issue Jan 6, 2022
Retry rename on EACCES

This PR retries `rename` upon getting `EACCES`. I've included data about how many retries are likely necessary.

On my system (WSL1 Ubuntu 20.04, omitting hardware details), the `EACCES` issue makes it impossible to use esy to install any of the Dream examples or use Dream's quick start. As the data below shows, every installation is expected to fail with `EACCES`, if it is not worked around.

The underlying `EACCES` issue seems to be a long-standing problem on WSL1 (microsoft/WSL#1529, microsoft/WSL#3395), and I think we do have to work around it in esy. I'm not sure what is causing the `EACCES` exactly. I think there are two main classes of possibilities:

- Self-interaction between esy's opened file descriptors and `rename`. I think the self-interaction is due to WSL rather than Lwt or another library. Since I compiled esy on WSL, it is using Lwt's Unix (rather than Windows) C code. Since the Unix code seems to work fine on Linux and Mac, this suggests a WSL issue.
- Interaction between esy and file indexers or other proceses running on the system. I'm not sure if that's a WSL issue or not, but I've never had to be aware of such processes when doing renames in Cygwin or elsewhere.

I built esy with this patch under WSL and ran clean `esy install`s in Dream's [full-stack ReScript](https://github.com/aantron/dream/tree/03e4d37cb5f5f638707479cd46105e2ee2b1df0e/example/w-fullstack-rescript#readme) example, using this script:

```sh
#!/bin/bash

export PATH="/home/antron/code/attic/esy/_build/install/default/bin:$PATH"
export ESY__PREFIX="/home/antron/code/dream/dream/example/w-fullstack-rescript/esy-prefix"
export OCAMLRUNPARAM=b
RUN=1

while true
do
  rm -rf esy-prefix _esy esy.lock lib node_modules/ package-lock.json
  echo
  echo "RUN $RUN"
  which esy
  esy install # --verbosity debug
  if [ $? != 0 ]
  then
    exit
  fi
  RUN=$((RUN+1))
done
```

The example was checked out into NTFS. The system was freshly restarted, and VSCode (or anything similar) was not running.

I used a version of this patch with a print showing the number of attempts before `rename` succeeds, and got the following results from 5 runs:

```
1 attempt:   802
2 attempts:   52
3 attempts:   12
4 attempts:    3
5 attempts:    1
total:       870
```

Based on this, I naively estimated that if a `rename` needs more than 1 attempt, the number of attempts needed decays by a factor of 4 at each step. I set the limit on the number of attempts naively to 8, thus expecting one failure in about 500 `esy install` attempts of the Dream ReScript example, under all these simplified assumptions.

The delay between attempts is (over) one second, so this means that upon legitimate `EACCES`, users will have to wait eight seconds to get an error message. I think there are two ways to address this:

- Fall back to recursive copy rather than retrying `rename` when `rename` fails. Do we have a recursive copy available in esy or its dependencies? Is it fine to leave the source directory intact?
- Detect WSL and retry only on WSL. Waiting for 8 seconds is still a much better user experience than failure to install at all, so the PR will still be an improvement, without, in this case, harming Linux or Mac users. We could also add a message, shown in case we finally fail with `EACCES` on WSL, giving users a hint about potential VSCode or other watchers, and what else they can try to solve the problem.

Closes #1363.
Probably fixes #1097, some of the reports after the first one.
Probably fixes #1083.
Probably fixes #593, but I haven't looked into non-WSL Windows yet.
Probably fixes aantron/dream#63.

cc @bryphe, @rizo, @jordwalke, @iMplode-nZ, @a-c-sreedhar-reddy, @srirajshukla, @andreypopp
@asimarora
Copy link

asimarora commented Jan 29, 2024

Hello,
I know the issue is closed but summarizing the full steps. I got the what but was not able to get how of it from this chat.
I am using WSL1 with Docker Desktop

File -> Preferences -> Settings
Click on the Open Settings(JSON) button on top right corner and change this "remote.WSL.fileWatcher.polling": true.

Hope I am not repeating someone else answer :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

10 participants