New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Very high CPU usage with new watcher #701

Closed
gaearon opened this Issue Jan 20, 2015 · 25 comments

Comments

Projects
None yet
@gaearon
Contributor

gaearon commented Jan 20, 2015

Now that 1.5 switched to new watcher by default, I noticed it's using a lot of CPU all the time:

screen shot 2015-01-20 at 14 30 14

It's not a Gulp issue because it goes away if I use OldWatchingPlugin.
It's also not related to compilation because it happens when Webpack is idle as well.

Any ideas?
OS X 10.10.1

@gaearon

This comment has been minimized.

Contributor

gaearon commented Jan 20, 2015

It doesn't seem to lag so much if I explicitly install fsevents so chokidar uses it.
Still, it's about 150% and doesn't get lower.

@sokra

This comment has been minimized.

Member

sokra commented Jan 20, 2015

@gaearon Is it lower for small projects? Maybe the watcher count is linear to CPU usage...

@gaearon Could you verify if fsevents is really used by chokidar?

fsevents should cause low cpu usage on OS X according to @paulmillr https://twitter.com/paulmillr/status/448600421054291970

Chokidar call: https://github.com/webpack/watchpack/blob/master/lib/DirectoryWatcher.js#L21-L27

@sokra sokra added the bug label Jan 20, 2015

@gaearon

This comment has been minimized.

Contributor

gaearon commented Jan 20, 2015

It's strange.
Even without fsevents, I have this issue in one project but not the other.

I'm not sure what the difference is yet.
I removed most of the source files but it's still 50% CPU, whereas other project works fine.

I'll keep investigating..

@gaearon

This comment has been minimized.

Contributor

gaearon commented Jan 20, 2015

OK, let's presume I did something stupid.
Nuking node_modules/webpack and npm install fixed the problem.

Still—I wonder if Watchpack (and Webpack) should now have fsevents as an optional dependency?
Otherwise it just doesn't get installed by default (even if it could be).
Or is this not how NPM works?

@gaearon gaearon closed this Jan 20, 2015

@sokra

This comment has been minimized.

Member

sokra commented Jan 20, 2015

chokidar has it as optional dep: https://github.com/paulmillr/chokidar/blob/master/package.json#L46

So theoretically it should be installed if possible when installing chokidar...

@paulmillr

This comment has been minimized.

paulmillr commented Jan 20, 2015

yeah everything is cool this way

@onpaws

This comment has been minimized.

onpaws commented Mar 8, 2016

In case this helps someone else, I was able to fix high CPU usage by doing what gaearon suggested in a previous comment.
I had restored a Time Machine backup from Laptop A, on which I previously ran npm install, to Laptop B.
On Laptop B, deleting node_modules/ and running npm install brought CPU usage from ~45% down to 0%.

OS X 10.11.3 (on both laptops).

@josephg

This comment has been minimized.

josephg commented May 1, 2016

I just ran into this too. You don't need to run npm install on your whole project again. Just npm install fsevents in the project directory.

@laggingreflex

This comment has been minimized.

laggingreflex commented May 24, 2016

Facing the same issue but on Windows 10, and I can't install fsevents, it's an OS X-only module...
edit: ignoring node_modules solved it for me

@mistercrunch

This comment has been minimized.

mistercrunch commented Jul 13, 2016

For the record, nuking node_modules/webpack and running npm install also fixed this for me.

@lexkrstn

This comment has been minimized.

lexkrstn commented Jul 29, 2016

What should I do on Windows 8.1 to solve the issue?

  • fsevents is an OSX-only module and I cannot install it on Windows.
  • Nuking node_modules/webpack and installing it again didn't help.

All the time webpack-dev-server is running it's process consumes about ~30% of CPU.

@gaearon

This comment has been minimized.

Contributor

gaearon commented Jul 30, 2016

@coderlex Have you tried this? webpack/watchpack#23

@lexkrstn

This comment has been minimized.

lexkrstn commented Jul 30, 2016

@gaearon I'm sorry for dusturbing. I've just found out the cause of my problem and it doesn't relate to this topic. The reason it consumed so much CPU was in these lines:

watchOptions: {
    aggregateTimeout: 300, // <---------
    poll: 1000, // <---------
    ignored: /node_modules/
},

I haven't even seen what they do in docs as the most of the config came from angular2-webpack-starter project. Commenting the lines solves the problem.

@winterbe

This comment has been minimized.

winterbe commented Oct 11, 2016

npm install fsevents fixes the issue, however putting fsevents into package.json breaks builds on other platforms, e.g. linux. So you have to remember that every time you rm -rf node_modules && npm install you also have to do npm install fsevents.

My brain is lazy so chances are good I totally forget about in a couple of weeks. 😭

@Kovensky

This comment has been minimized.

Collaborator

Kovensky commented Oct 11, 2016

chokidar (the indirect dependency that depends on fsevents) will install it on its own, an an optionalDependency.

The only case fsevents may not be picked up is if you're using a shrinkwrap generated in a non-mac machine. OTOH, if you do use a shrinkwrap generated in a mac, npm install itself will fail on non-macs, so pick your poison.

@onpaws

This comment has been minimized.

onpaws commented Oct 11, 2016

@winterbe I feel the same way about keeping dependencies obvious. Why not make a note in your package.json?

Because JSON doesn't support comments some folks have come up with alternative approaches.

@ngyikp

This comment has been minimized.

ngyikp commented Oct 27, 2016

In my experience, this CPU hogging happens every time I upgrade node.js to a new version, I need to reinstall fsevents (or just nuke the node_modules folder and reinstall)

@Kovensky

This comment has been minimized.

Collaborator

Kovensky commented Oct 27, 2016

@ngyikp in that case, you just need npm rebuild.

chitacan added a commit to riiid/riiiact-kit that referenced this issue Nov 7, 2016

@msmfsd

This comment has been minimized.

msmfsd commented Nov 24, 2016

Thanks @gaearon for this fix, saved my day 👍

@maartenscholz

This comment has been minimized.

maartenscholz commented Dec 1, 2016

If you use yarn you can do yarn add fsevents --optional. This will allow you to save fsevents in package.json but it will not break builds on other platforms.

Optional dependencies are just that: optional. If they fail to install, Yarn will still say the install process was successful.
This is useful for dependencies that won’t necessarily work on every machine and you have a fallback plan in case they are not installed (e.g. Watchman).

@chemzqm

This comment has been minimized.

chemzqm commented Dec 21, 2016

I should keep a note for npm rebuild

@GotJimmy

This comment has been minimized.

GotJimmy commented May 16, 2017

Deleting the entire node_modules folder and npm install fixed the problem for me. Just nuking the webpack folder did not do the trick.

billyvg added a commit to getsentry/sentry that referenced this issue Jun 28, 2017

Adds fsevents as optional dependency
This should solve webpack using up 100% CPU on OSX.

See webpack/webpack#701

Note fsevents is an OSX only package, but having it as an "optional"
dependency means yarn will ignore build errors for the package.
@hosembafer

This comment has been minimized.

hosembafer commented May 15, 2018

Solved by simply running npm install fsevents

@davideschiera davideschiera referenced this issue Aug 8, 2018

Closed

CPU Usage #752

@danburzo

This comment has been minimized.

danburzo commented Aug 28, 2018

Is this still relevant? If so, should we add a canonical solution to the docs?
I've done this:

yarn add --optional fsevents
rm -rf node_modules
yarn

and it does seem to help, but I'd love to know if these commands are actually doing anything useful or it's just magical thinking :-)

@tkluysk

This comment has been minimized.

tkluysk commented Sep 11, 2018

150% in watch mode using webpack 4.17.2.

image

The fsevents fix did not work for me, but this did:
https://webpack.js.org/configuration/watch/#watchoptions-ignored

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment