Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WSL2] File changes made by Windows apps on Windows filesystem don't trigger notifications for Linux apps #4739

Open
SteveSandersonMS opened this issue Dec 6, 2019 · 137 comments
Labels
feature wsl2 Issue/feature applies to WSL 2

Comments

@SteveSandersonMS
Copy link

WSL2 is really close to being a perfect runtime environment for server apps being developed in Windows. Great job! One missing feature however is breaking a core part of the developer flow.

For sources stored on the Windows filesystem, any changes made by Windows applications such as Visual Studio do not trigger any file change notifications as far as Linux apps are concerned. This means that all "live rebuild"-type tools don't work (examples: webpack --watch, jekyll --interactive, and Tilt.dev) when running under WSL2. This unfortunately renders many modern dev workflows unviable.

Notes:

Bug report template

  • Your Windows build number: 10.0.19033.1

  • What you're doing and what's happening:

    This applies to all tools that listen for file change notifications, but as an example take webpack. Repro steps:

    • In your Windows filesystem, create an empty directory (example: c:\repro), and then add these three files to it
    • In a WSL2 Ubuntu 18.04 environment, install Node and NPM: sudo apt-get install nodejs npm
    • Still in WSL2, go into the directory from earlier: cd /mnt/c/repro
    • Restore NPM dependencies: npm i
    • Run Webpack in watch mode: npm run build:watch. Wait a few seconds until it completes the first build. It will now be waiting for further changes to your source files.
    • In a Windows application (e.g., Notepad or Visual Studio), open c:\repro\index.js and save some change to it. For example, change 'Hello, world' to 'Hello, world 2'.
  • What's wrong / what should be happening instead:

    Expected behavior: Webpack should see the change and rebuild. That is, you'll see it log information about another build, and the output in dist/bundle.js will be updated.

    Actual behavior: Webpack doesn't respond at all, because there's no file change notification.

Finally I understand that the fix for this is likely to be "add file watch capabilities to the Plan9 server", and you may feel this is already being tracked by #4064. However #4064 describes a more obscure symptom of this missing feature and makes it sound like an intermittent issue. What I'm reporting here is not intermittent at all, and is a pretty mainstream scenario (using tools like webpack --watch). Thanks!

@therealkenc
Copy link
Collaborator

therealkenc commented Dec 6, 2019

Yes, #4064 is most unfortunately titled. Worse, the OP has no repro steps and an unecessary symlink confusing the direction they are talking about. Craig's notepad.exe repro with cat isn't an inotify(7) (file watcher) problem, even if implementing inotify is the fix to whatever loosly implied sync problems are going on over there.

Nevertheless #4701 got closed as a dupe with (quoth):

We need to add file watch capabilities to the Plan9 server that serves files to a WSL2 distro, and we're tracking that work item here

So, best I can tell, #4064 is being treated as the LZ for inotify triggers from Windows to WSL2. For lack. #4169 is pretty much exactly your use-case also landed dupe #4064.

This did work great on WSL1

Yes, known regress.

@nake89
Copy link

nake89 commented Jan 4, 2020

Is there a temporary fix to this? This is giving me a headache.

Using Windows 10, WSL 2. Running npm run serve on my Vue project, which normally hot reloads when I do changes (on my Linux and Mac, and WSL1 and I think maybe WSL2 before(?)). Now I have to shut down and restart the Vue server.

Will probably just install Linux on this machine. Developing on a Windows machine is a real pain.

@craigloewen-msft
Copy link
Member

@nake89 are you able to move your project over to the Linux root file system? i.e: Store in your Linux home folder for example? Are there any factors that are blocking you from doing so?

And per @therealkenc 's comment this would be a much better landing zone for adding inotify to the 9P file server. I'll update my comment in #4701 to point here, and add some tags to this issue.

@nake89
Copy link

nake89 commented Jan 7, 2020

@craigloewen-msft Thanks for messaging me back! I actually solved my issue by moving my project to the linux filesystem in WSL and have had no problems so far. I suggest doing the same. In VS Code simply type ctrl-shift-p and then type "Remote-WSL: New Window" This lets users use the linux filesystem in vs code, for those that did not know this :)

@SteveSandersonMS
Copy link
Author

SteveSandersonMS commented Jan 7, 2020

@nake89 That's great if your scenario allows it. But just to clarify for anyone else reading, that's not a solution in general as it doesn't work for other Windows-based editors such as VS.

@craigloewen-msft
Copy link
Member

@SteveSandersonMS agreed, we will still be tracking this issue here. :)

@ghost
Copy link

ghost commented Jan 23, 2020

Hi, I am encountering this with docker-sync. My setup is:

  • working folder under C:\users for various php projects
  • Edit a file in phpstorm
  • Changes reflected within my WSL2 Ubuntu instance
  • Running docker containers within WSL2 with volumes synced using docker-sync and Unison

If I make a change to a file directly inside Ubuntu, changes are reflected in the container's mounted volume. However, if I edit a file in PhpStorm in Windows, the change is reflected in Ubuntu as expected but not in the container.

I can't easily move my files to the Linux filesystem because I need to use PhpStorm and other Windows tools in my dev environment, and I also have other stuff such as Google drive sync running to back up local changes. So the workaround doesn't work for me.

@Carl-Hugo
Copy link

@craigloewen-msft Thanks for messaging me back! I actually solved my issue by moving my project to the linux filesystem in WSL and have had no problems so far. I suggest doing the same. In VS Code simply type ctrl-shift-p and then type "Remote-WSL: New Window" This lets users use the linux filesystem in vs code, for those that did not know this :)

This is a great workaround that will save me lots of time until WSL2 supports this use-case. Thanks for sharing!

@Carl-Hugo
Copy link

Carl-Hugo commented Feb 5, 2020

@craigloewen-msft Thanks for messaging me back! I actually solved my issue by moving my project to the linux filesystem in WSL and have had no problems so far. I suggest doing the same. In VS Code simply type ctrl-shift-p and then type "Remote-WSL: New Window" This lets users use the linux filesystem in vs code, for those that did not know this :)

This is a great workaround that will save me lots of time until WSL2 supports this use-case. Thanks for sharing!

I wrote a small blog post explaining this workaround and talking about a few more things that I discovered/experienced, using Jekyll on WSL2: Speed up your builds to up to 375% and watch for changes for an even faster dev cycle using this workaround on WSL2/Ubuntu, if that can help someone.

Note: that applies to more than just Jekyll, Jekyll it was just the catalyst.

@safizn
Copy link

safizn commented Feb 5, 2020

For those stumbling upon, some notes about WSL2 features & limitations (OS build 19041.21, insiders slow ring):

  • inotify filesystem events are not propagated between WSL2 & Windows. Although it will be supported in future releases as stated in Microsoft documentation.
  • Accessing Windows filesystem from WSL2, when developing is extremely slow. While moving projects to WSL2 filesystem, will increase performance, much faster than WSL1 & Windows development. (WSL2 can be accessed in path \\wsl$\)
  • VSCode installed in Windows, with remote extension pack, will install VSCode server automatically in WSL2. Some extensions should be installed in the WSL2 side to work, when openning files from the local WSL2 filesystem. The VSCode Extenions tab in the UI provides indications and guides through the required changes.
  • localhost is managed for you by Windows, and allows access to WSL2. Some cases require accessing the WSL2 VM by it's IP directly.
  • Symlinks in WSL2 work seamlessly between WSL2, Docker containers, & Windows, which wasn't the case with WSL1. Using Docker Desktop on WSL2 experimental feature.
  • Some graphical programs (e.g. SmartGit/GitKraken) need to be installed in WSL2 and accessed through GUI client through Windows (Unix X server), to overcome the inotify events & performance degredations.

@ivellios
Copy link

ivellios commented Mar 9, 2020

While I am having similar issue with my React App and I am looking for a solution to this as well, I can reply to:

I can't easily move my files to the Linux filesystem because I need to use PhpStorm and other Windows tools in my dev environment, and I also have other stuff such as Google drive sync running to back up local changes. So the workaround doesn't work for me.
@weknowsoftware

@nake89 That's great if your scenario allows it. But just to clarify for anyone else reading, that's not a solution in general as it doesn't work for other Windows-based editors such as VS.
@SteveSandersonMS

This hint by @myuseringithub:

  • Accessing Windows filesystem from WSL2, when developing is extremely slow. While moving projects to WSL2 filesystem, will increase performance, much faster than WSL1 & Windows development. (WSL2 can be accessed in path \\wsl$\)

is a great one to help you! Also I found out yesterday that since \\wsl$\ is a network resource in Windows you can easily map it to a drive. Then in ANY editor/tool you can use it as if it was on your computer. Just go to the Explorer and manually enter this address to access the resource. You will most likely see "Ubuntu" folder. It is a root of your WSL and it can be mapped to the drive (right click -> map to drive). That helped me setup my project on WSL natively, while being able to edit in the windows editor.

Though now I am struggling with Docker containers permissions, which I hope to solve separately and see if reloads in my app work.

Hope that helps.

@nicks
Copy link

nicks commented Mar 27, 2020

Is there anything that app devs can do to workaround this in their apps? e.g., is there a different file-change notification API that would work on WSL2 across filesystems? (there are a lot of different file-change notification APIs). Or should we wait patiently for inotify support?

@Carl-Hugo
Copy link

Is there anything that app devs can do to workaround this in their apps? e.g., is there a different file-change notification API that would work on WSL2 across filesystems? (there are a lot of different file-change notification APIs). Or should we wait patiently for inotify support?

You can use the Linux file system directly, accessible from Windows too; see #4739 (comment)

@mattlacey
Copy link

@Carl-Hugo Does that work for you, as in, are changed detected? I've been using the \wsl$ path with windows editors for a while because with WSL2 the Linux FS is so much faster, but even when using windows editors with files stored there it's not triggering HMR when using npm watch.

@Carl-Hugo
Copy link

@mattlacey Yes it works...

  1. image
  2. ctrl+s the README.md file in VS Code (Windows)
  3. image

Side note: for-the-new-order is not a sect but a Star Wars RPG thing 😉

@vielhuber
Copy link

Some graphical programs (e.g. SmartGit/GitKraken) need to be installed in WSL2 and accessed through GUI client through Windows (Unix X server), to overcome the inotify events & performance degredations.

Can you please elaborate on that: Is this working well? Any tutorials / starting points how to set this up?

@safizn
Copy link

safizn commented Apr 17, 2020

@vielhuber I wrote it as a comment for myself, tried to implement it once without success. Just wait till WSL2 will support inotify. You could still use graphical interfaces for git, only that you would have to constantly refresh to see changes, which is the same case when dealing with VSCode's git panel.

But if you wish to dig deeper, check out:

I wrote these comments when setting up my development environment

@vielhuber
Copy link

@myuseringithub Awesome thank you.

@huysentruitw
Copy link

I use my regular Git client in Windows, because my project is still on my Windows machine. However, to get livereload working, I simply add a symlink from my WSL2 home directory to the project on my windows machine, f.e.:

ln -s /mnt/d/projects/contrib/create-your-future-website/ ~/create-your-future-website

After that, I start jekyll serve --livereload from ~/create-your-future-website and open that folder in VS code using ctrl-shift-p and then type "Remote-WSL: New Window" as explained above.

This way, you don't have to move your project and can still enjoy your favorite git client in Windows. Profit!

@huysentruitw
Copy link

Arg, scratch the above idea. It only seems to get triggered by file changes at the top-level directory. 😢

@omt66
Copy link

omt66 commented Oct 13, 2023

Can someone lock this thread, please? Getting tired of "me too" notifications, and it's fairly clear the maintainers aren't going to respond, so you might as well block this noise and just let us know if it ever gets fixed.
Or if you're not going to address this, maybe just close the thread.
Everyone knows what the problem is, and only MS can solve it - no reason to waste anybody else's time then.

Umm click unsubscribe? Me too comments demonstrate how widespread the issue is and allow for discussion around workarounds/root cause. If you're not interested, simply unsubscribe.

Ditto. This is a big problem, if anyone doesn't want to hear, they should unsubscribe!

@adam-ah
Copy link

adam-ah commented Feb 16, 2024

Geez, 4 years, still no solution? This bug breaks so many things.
How do we escalate it to MS to have a look?

@PaulOst
Copy link

PaulOst commented Feb 17, 2024

Is MS monitoring these issues?

@erentar
Copy link

erentar commented Feb 24, 2024

I also desperately need this feature

@mattbell87
Copy link

mattbell87 commented Feb 24, 2024

I also desperately need this feature

Can you run your project inside WSL? That fixes issues like this for me. If you need access from Windows just use \\wsl$ in explorer/files.

@raffaeler
Copy link

Can you run your project inside WSL? That fixes issues like this for me. If you need access from Windows just use \\wsl$ in explorer/files.

accessing is not enough, the need is to use rsync to keep the files updated (aka file system notification).

Anyway, on my side I don't care anymore as I moved my dev machine to Ubuntu and resolved this issue radically. The best surprise from this move, was discovering the file system operations on Ubuntu are hugely faster during C++/.NET compile time or nodjs development.

@EmmanuelTheCoder
Copy link

EmmanuelTheCoder commented Feb 26, 2024

I also desperately need this feature

Can you run your project inside WSL? That fixes issues like this for me. If you need access from Windows just use \\wsl$ in explorer/files.

That works but it defeats interoperability. Didn't they say that WSL is interoperable with Windows?

@MiikaH
Copy link

MiikaH commented Feb 26, 2024

That works but it defeats interoperability. Didn't they say that WSL is interoperable with Windows?

Not sure what you mean by that.

You keep your projects in WSL file system, access them via \\wsl.localhost\ . Explorer access, editor & IDE access, docker and everything should pretty much work the same way as if you stored your project on C:\, D:\ etc, but now the files are on Linux ext4 filesystem so performance should be way higher than NTFS and file system notifications work as expected, as stated above.

And if you want to take things one step further you can (on top of WSL) setup your project to use VSCode "Dev Containers" in order to run the project & VSCode in virtual Linux distro of your choosing. Which not only makes it easier to run Linux primary projects on Windows as you can just run the linux commands natively for the project, improves WSL disk compatibility when the whole setup is doing native linux filesystem calls, and it also separates project dependencies of you main system. As in you can run whichever version of Node.js, Python for each project dev container without any conflicts with versions installed on your host OS. That way if project x recommends Debian, Python 3.10, Cmake 3.8 or whatever - no problem, just configure your devcontainer with those dependencies and you are good to go.

@adam-ah
Copy link

adam-ah commented Feb 26, 2024

@MiikaH sorry, this is totally irrelevant. Could we please stay on topic and focus on the bug in WSL?
(this message does not require a response)

@MiikaH
Copy link

MiikaH commented Feb 26, 2024

Discussing workarounds for the bug is the only relevant use for this topic, so that people who are affected can find a working solution for their use case. Especially when it is clear that we may have to wait decade or two until Microsoft actually fixes the bug itself.

@HugoImaios
Copy link

HugoImaios commented Feb 26, 2024

You can't call "don't use X feature" a workaround for an issue titled "feature X does not work".

The issue is "File changes made by Windows apps on Windows filesystem don't trigger notifications for Linux apps".
Therefore, "don't use Windows filesystem, use the Linux one", does not seem an acceptable workaround for this particular issue (even if it actually works, as much as installing a full ubuntu, running a rapsberry or riding a bike)

@WaldemarH
Copy link

Nothing works :(

I've tried the /mnt/... version it doesn't work.

I've also tried the linux filesystem:

  1. I've created /var/www/client folder
  2. mounted "net use Y: \wsl.localhost\RockyLinux-8 /PERSISTENT:YES"
  3. started webpack in client
  4. edited a file on a Y disk inside the client folder and nothing, zero, nada, zilch

So currently WSL2 is useles for webpack development. :(

@mcnilz
Copy link

mcnilz commented Mar 26, 2024

I found a pretty ugly workaround some time ago. Since the pain of this issue seems never to end I want to share it.

In my development docker image I added the "Unison file synchronizer", supervisor and scripts.
Unison is used to synchronize the to /tmp directory (that must be volume or tmpfs but not bind).
Supervisor is used to manage the ongoing sync process and other goodies out of the scope of the main issue.

The code for nodejs development is here: https://github.com/bridgefield/docker-node-dev
Dotnet development here https://github.com/bridgefield/dotnet-core-sdk-extra
The repositories are rarely updated sinice they are not in use at the moments. Maybe ppl can adopt the ideas to their projects.

The solution is far from being perfect. It takes much performance because unison can not use ionotify.

@adam-ah
Copy link

adam-ah commented Mar 27, 2024

@WaldemarH maybe it's better to delete your not-so-relevant comments on this bug? This is a bug report about WSL2 file system change notifications, not a samba tutorial.

@rw3iss

This comment has been minimized.

@WaldemarH
Copy link

WaldemarH commented Mar 27, 2024

@rw3iss So should I put it back on? .. I know I would be happy if somebody gave me at least some working option.

@rw3iss
Copy link

rw3iss commented Mar 27, 2024

@WaldemarH Yeah... would suggest just summarizing it a bit as you did, as an alternative solution, and then maybe linking to a gist with full instructions. Doesn't hurt and someone else may find it useful for sure. AFAIK it's one of the only ways to actually link them via network operations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature wsl2 Issue/feature applies to WSL 2
Projects
None yet
Development

No branches or pull requests