This repository has been archived by the owner. It is now read-only.

FileSystemWatcher issues when running dnx-watch on Mono #13

Closed
miguellira opened this Issue Sep 29, 2015 · 32 comments

Comments

Projects
None yet
7 participants
@miguellira
Copy link

miguellira commented Sep 29, 2015

Running dnx-watch against a simple web application on Mono works fine.

However, given the known limitations of FileSystemWatcher in Mono (aspnet/AspNetCore#508, aspnet/AspNetCore#554) the only way I've managed to run a reasonably sized web application is to set MONO_MANAGED_WATCHER="disabled"

Having this environment variable set appears to disables dnx-watch, however, unsetting the variable produces the following error:

Miguels-MBP:WebApp miguellira$ dnx-watch kestrel
[DnxWatcher] info: Running dnx with the following arguments: --project /Users/miguellira/Sandbox/WebApp/project.json kestrel
[DnxWatcher] info: dnx process id: 8610
/Users/miguellira/.dnx/runtimes/dnx-mono.1.0.0-beta8-15736/bin/dnx: line 9: pwd: write error: Broken pipe
System.IO.IOException: kqueue() FileSystemWatcher has reached the maximum number of files to watch.
   at System.IO.KqueueMonitor.Start () in <filename unknown>:line 0
   at System.IO.KeventWatcher.StartDispatching (System.IO.FileSystemWatcher fsw) in <filename unknown>:line 0
   at System.IO.FileSystemWatcher.Start () in <filename unknown>:line 0
   at System.IO.FileSystemWatcher.set_EnableRaisingEvents (Boolean value) in <filename unknown>:line 0
   at (wrapper remoting-invoke-with-check) System.IO.FileSystemWatcher:set_EnableRaisingEvents (bool)
   at Microsoft.Dnx.Watcher.Core.FileWatcher.AddWatcher (System.String path) in <filename unknown>:line 0
   at Microsoft.Dnx.Watcher.Core.FileWatcher..ctor (System.String path) in <filename unknown>:line 0
   at Microsoft.Dnx.Watcher.Core.DnxWatcher+<>c.<CreateDefault>b__13_0 (System.String root) in <filename unknown>:line 0
   at Microsoft.Dnx.Watcher.Core.DnxWatcher+<WaitForProjectFileToChangeAsync>d__7.MoveNext () in <filename unknown>:line 0

Is dnx-watch not recommended for use with Mono until that bug is addressed?

@muratg

This comment has been minimized.

Copy link
Member

muratg commented Oct 2, 2015

@victorhurdugaci is this known issue in Mono? If so, can we track it in the External repo, and close this one?

cc @moozzyk

@victorhurdugaci

This comment has been minimized.

Copy link
Contributor

victorhurdugaci commented Oct 2, 2015

It is a known mono issue: https://bugzilla.xamarin.com/show_bug.cgi?id=28693

@miguellira how many files does your app have?

@miguellira

This comment has been minimized.

Copy link

miguellira commented Oct 2, 2015

@victorhurdugaci Not many. I haven't tried to figure out the limit but as you know npm modules tend to have huge directory trees.

For now, I simply delete the node_modules, bower_components, and tidy up my target lib folder via a gulp task post restore and I can run dnx-watch.

@muratg

This comment has been minimized.

Copy link
Member

muratg commented Oct 5, 2015

@miguellira I'm glad you're able to work around it although it's not an ideal solution. Feel free to push the Mono bug to resolution :)

@victorhurdugaci Cool, please file an External to track the Mono bug and close this one.

@nathanielcook

This comment has been minimized.

Copy link

nathanielcook commented Dec 4, 2015

In my case it was the .git folder that was causing the problem. I moved my code into a src folder and ran dnx-watch there instead of the main root, and that works fine (since src doesn't include the .git folder).

@victorhurdugaci

This comment has been minimized.

Copy link
Contributor

victorhurdugaci commented Dec 4, 2015

@nathanielcook not sure how that would cause an issue. Can you give more details about the folder structure that you have before it started working and what command were you running?

@nathanielcook

This comment has been minimized.

Copy link

nathanielcook commented Dec 4, 2015

Before

projectroot
  .git
  Controllers
  Models
  Views
  project.json
  project.lock.json
  Properties
  Startup.cs
  wwwroot
  LICENSE
  README.md

After

projectroot
  .git
  LICENSE
  README
  src
    Models
    Controllers
    Views
    etc

Before I was running dnx-watch in projectroot. Now I am running dnx-watch in projectroot/src. Make sense?

@victorhurdugaci

This comment has been minimized.

Copy link
Contributor

victorhurdugaci commented Dec 4, 2015

ohhh... yes, that could be a problem on Mono because of the limit that @miguellira mentioned. The .git folder could have lots of files in it.

@victorhurdugaci

This comment has been minimized.

Copy link
Contributor

victorhurdugaci commented Dec 4, 2015

Btw, can you try running on CoreCLR?

@nathanielcook

This comment has been minimized.

Copy link

nathanielcook commented Dec 4, 2015

I'm not sure what that means.

@victorhurdugaci

This comment has been minimized.

Copy link
Contributor

victorhurdugaci commented Dec 4, 2015

dnx-watch runs on the same CLR as your project. On Linux we have Mono and CoreCLR. If your project supports CoreCLR (aka it has the dnxcore50 target) you can try running on it. You shouldn't hit the file limit.

Instructions on how to install CoreCLR on Linux: https://docs.asp.net/en/latest/getting-started/installing-on-linux.html

Then dnvm use -r CoreCLR

And then the watcher will use the active CoreCLR runtime.

@miguellira

This comment has been minimized.

Copy link

miguellira commented Dec 4, 2015

@nathanielcook Yeah, you should always consider initializing git on your project root. BTW, as soon as you exceed 200 files/folders in the folder you run dnx-watch web it will barf.

It all boils down to this line in Mono:
https://github.com/mono/mono/blob/b7a308f660de8174b64697a422abfc7315d07b8c/mcs/class/System/System.IO/KeventWatcher.cs#L444

It seems as though 200 is the magic #:
https://github.com/mono/mono/blob/b7a308f660de8174b64697a422abfc7315d07b8c/mcs/class/System/System.IO/KeventWatcher.cs#L641

I'm sure the Mono guys know why since that bug is still outstanding.

@miguellira

This comment has been minimized.

Copy link

miguellira commented Dec 4, 2015

@victorhurdugaci For the record, dnx-watch indeed runs without limitation on dnxcore50. Unfortunately, if you have a dependency that requires dnx451 you're stuck with mono and the 200 file limit.

@muratg

This comment has been minimized.

Copy link
Member

muratg commented Dec 5, 2015

@miguellira Could you add this information to the Mono bug report (if it's not already there)?

I suspect they haven't investigated this bug yet, but since you already seem to have found the root cause, I'm sure they would appreciate the input :)

@miguellira

This comment has been minimized.

Copy link

miguellira commented Dec 5, 2015

@muratg Done. I've updated the bug with my findings.

@muratg

This comment has been minimized.

Copy link
Member

muratg commented Dec 7, 2015

Closing this one as this is a Mono issue and there's no action on our side.

@muratg muratg closed this Dec 7, 2015

@davidfowl

This comment has been minimized.

Copy link
Member

davidfowl commented Dec 7, 2015

DNX watch should only be compiled for dnxcore50 moving forward

@miguellira

This comment has been minimized.

Copy link

miguellira commented Dec 7, 2015

@davidfowl Though that would be clear to all dnx-watch new comers those who still depend on dnx4xx and are aware of the limitation lose out.

@davidfowl

This comment has been minimized.

Copy link
Member

davidfowl commented Dec 7, 2015

It doesn't matter what your application depends on. The tool itself can target whatever. It's just restarting a process

@miguellira

This comment has been minimized.

Copy link

miguellira commented Dec 7, 2015

@davidfowl I was under the impression that dnx-watch ran using the current active runtime. If my app depends on a dnx451 component requiring mono, would dnx-watch compiled for dnx50core still run?

I'm sorry, I'm sure I'm missing something.

@miguellira

This comment has been minimized.

Copy link

miguellira commented Dec 9, 2015

Since Miguel de Icaza lifted the "arbitrary" 200 file limit as of this commit, this issue should go away.

My mono is delivered via homebrew so I'll wait until that bottle is updated to test.

That said, I'm still curious about @davidfowl suggestion that a dnx50core compiled dnx-watch would still be applicable in a dnx4xx executing app.

@victorhurdugaci

This comment has been minimized.

Copy link
Contributor

victorhurdugaci commented Dec 9, 2015

It's just a matter of what exe we start from the watcher. If we launch the one from dnx-clr then the app will run on Desktop CLR and if we launch the one in dnx-coreclr the app will run on CoreCLR.

@muratg

This comment has been minimized.

Copy link
Member

muratg commented Dec 9, 2015

@victorhurdugaci When we move this to dotnet CLI, it'll only be a CoreCLR based exe. And it's going to just start the app itself (can be CLR or CoreCLR)

@jasoncavett

This comment has been minimized.

Copy link

jasoncavett commented Feb 19, 2016

@miguellira I just ran into this issue. Thanks for sharing information about the fix. It appears that this made it into homebrew already (and I have 4.2.2.30 release installed), but the issue is still occurring. Am I missing something with regards to this?

@miguellira

This comment has been minimized.

Copy link

miguellira commented Feb 19, 2016

@jasoncavett Hate to respond with a "it works on my machine" but I'm not sure why you're running into this issue. It was fixed with the mono patch I mentioned above and I confirmed it with a custom build of mono at the time.

FWIW, I just updated mono via brew to 4.2.2.30 and re-tested against the latest yo aspnet template and did not run into any issues.

[DnxWatcher] info: File changed: /Users/miguellira/Sandbox/WebApp/Controllers/HomeController.cs
[DnxWatcher] info: Running dnx with the following arguments: --project /Users/miguellira/Sandbox/WebApp/project.json web
[DnxWatcher] info: dnx process id: 10117
[DnxWatcher] info: File changed: /Users/miguellira/Sandbox/WebApp/Controllers/HomeController.cs
[DnxWatcher] info: Running dnx with the following arguments: --project /Users/miguellira/Sandbox/WebApp/project.json web
[DnxWatcher] info: dnx process id: 10125
Hosting environment: Development
Now listening on: http://localhost:5000

Might be worth re-installing dnx-watch. Good luck.

@nathanielcook

This comment has been minimized.

Copy link

nathanielcook commented Feb 19, 2016

@jasoncavett

This comment has been minimized.

Copy link

jasoncavett commented Feb 19, 2016

@miguellira Yeah, I figured that might be the answer. I did a complete uninstall of everything and reinstall with no luck. I do have two flags set in my .bash_profile that I had disabled - one being export MONO_MANAGED_WATCHER=disabled which is what stopped everything from breaking in the first place. When I comment it out, though, I get the error that I am watching too many files still once I actually hit my site's endpoint. Everything does start up fine, however.

To your point @nathanielcook - I figured that might fix it, but, I was under the impression that the error was fixed. Not to mention that I'm running a across multiple projects, so I dnx restarted when any of those files change so I can't easily watch at the src folder.

I'm running on El Capitan.

mcs --version
Mono C# compiler version 4.2.2.0

brew info mono
mono: stable 4.2.2.30 (bottled)
Cross platform, open source .NET development framework
http://www.mono-project.com/
Conflicts with: czmq, disco, xsd
/usr/local/Cellar/mono/4.2.2.30 (1,080 files, 205.1M) *
  Poured from bottle
From: https://github.com/Homebrew/homebrew/blob/master/Library/Formula/mono.rb

dnx-watch Version is 1.0.0-rc1-final

I'll keep playing around with it and see what I can come up with. Thank you both of you for responding.

@jasoncavett

This comment has been minimized.

Copy link

jasoncavett commented Feb 19, 2016

I did just discover something interesting. Looking into the FileSystemWatcher.cs at line 108 there is a reference to the MONO_MANAGED_WATCHER environment variable. Seeing that any value (other than disabled which is handled lower) causes it to use the DefaultWatcher, I just switched my environment variable to be enabled. This half works. The reason I say half is because, it does recognize changes to my files, but very slowly, and it's cranking away on my CPU. It's not a viable solution, of course, but, found it interesting.

@miguellira

This comment has been minimized.

Copy link

miguellira commented Feb 19, 2016

@jasoncavett Sorry about that, I should have been clearer. You must keep the MONO_MANAGED_WATCHER enviornment variable set to disabled. When I opened this issue it was due to the hard coded 200 limit, however, if you're running against Mono you must have that environment variable set otherwise you'll get the "too many files open" error.

BTW, does your application taking any dependencies requiring dnx451? Might want to just stick to dnxcore for now. Also, with dotnet-cli out in beta you might want to start looking at what issues your app will run into with the latest ASPNETCore bits.

@jasoncavett

This comment has been minimized.

Copy link

jasoncavett commented Feb 19, 2016

@miguellira Ahhh...okay. I want to enable file watching because I'm looking to have dnx-watch restart kestrel as I am coding. Otherwise, it kind of defeats the purpose of having it, no?

Right now, I have at least one known library that still has a dependency on dnx451. I'm hoping to get away from that, but, it would be a fairly big change since it was used heavily and brought over from some legacy code. I didn't realize dotnet-cli was out of beta. Is this basically replacing dnu?

Thanks again for your help with everything.

@miguellira

This comment has been minimized.

Copy link

miguellira commented Feb 19, 2016

@jasoncavett Again, dnx-watch does restart kestrel as you're coding. The console snapshot I posted above #13 (comment) was caused by touching the HomeController.cs file.

To be clear dotnet-cli is not out of beta but yes, the idea is that all the dnx/dnu commands will be replaced by a single CLI. dnx-watch will eventually be ported over as well.

@victorhurdugaci

This comment has been minimized.

Copy link
Contributor

victorhurdugaci commented Feb 19, 2016

Dnx watch was ported to dotnet :)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.