Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

High CPU usage #35

Closed
flynfish opened this issue Jan 9, 2013 · 38 comments
Closed

High CPU usage #35

flynfish opened this issue Jan 9, 2013 · 38 comments

Comments

@flynfish
Copy link

@flynfish flynfish commented Jan 9, 2013

When running grunt watch, the process sits around 70% cpu usage on my mac (10.8.2). Is this a known issue?

@shama
Copy link
Member

@shama shama commented Jan 9, 2013

It's not. How many files are you watching and could you post your Gruntfile.js please? Does it happen with just watching or as tasks are ran? Also are you using grunt@0.4.0rc4 and grunt-contrib-watch@0.2.0a?

@flynfish
Copy link
Author

@flynfish flynfish commented Jan 9, 2013

Versions:
"grunt": "~0.3.17",
"grunt-contrib-watch": "~0.1.4",
"grunt-contrib": "~0.3.0"

As for the amount of files, since I am using bootstrap there are more 40 less files being watched. There really isn't much of a need for me to watch those though so perhaps that is one of the issues? Looks like it happens just when watching.

module.exports = function(grunt) {

  grunt.initConfig({

    // The lint task will run the build configuration and the application
    // JavaScript through JSHint and report any errors.  You can change the
    // options for this task, by reading this:
    // https://github.com/cowboy/grunt/blob/master/docs/task_lint.md
    lint: {
      files: [
        "public/**/*.js"
      ]
    },

    // The jst task compiles all application templates into JavaScript
    // functions with the underscore.js template function from 1.2.4.  You can
    // change the namespace and the template options, by reading this:
    // https://github.com/tbranyen/build-tasks/tree/master/jst
    //
    // The concat task depends on this file to exist, so if you decide to
    // remove this, ensure concat is updated accordingly.
    jst: {
      "public/js/templates.js": [
        "views/templates/**/*.html"
      ]
    },

    // The concatenate task is used here to merge the almond require/define
    // shim and the templates into the application code.  It's named
    // dist/debug/require.js, because we want to only load one script file in
    // index.html.
    concat: {
      "public/js/debug.js": [
        "public/js/vendor/*.js",
        "public/js/vendor/bootstrap/*.js",
        "public/js/templates.js",
        "public/js/app/*.js",
        "public/js/app/models/*.js",
        "public/js/app/views/*.js",
        "public/js/app/collections/*.js",
        "public/js/app/views/nests/*.js"
      ]
    },

    // This task uses the MinCSS Node.js project to take all your CSS files in
    // order and concatenate them into a single CSS file named index.css.  It
    // also minifies all the CSS as well.  This is named index.css, because we
    // only want to load one stylesheet in index.html.
    mincss: {
      "public/dist/release.css": [
        "public/stylesheets/*.css"
      ]
    },

    // Takes the built require.js file and minifies it for filesize benefits.
    min: {
      "public/dist/release.js": [
        "public/js/debug.js"
      ]
    },

    // less to CSS
    less: {
      compile: {
        files: {
          "public/stylesheets/style.css": "public/stylesheets/style.less",
          "public/stylesheets/nest.css": "public/stylesheets/nest.less",
          "public/stylesheets/style.css": "public/stylesheets/style.less",
          "public/stylesheets/vendor/bootstrap/bootstrap.css": "public/stylesheets/vendor/bootstrap/less/bootstrap.less",
          "public/stylesheets/vendor/fontawesome/font-awesome.css": "public/stylesheets/vendor/fontawesome/less/font-awesome.less"
        }
      }
    },

    watch: {
      scripts: {
        files: '**/*.less',
        tasks: ['less']
      },
      templates: {
        files: 'views/templates/**/*.html',
        tasks: ['jst']
      }
    }

  });

  // The default task will remove all contents inside the dist/ folder, lint
  // all your code, precompile all the underscore templates into
  // dist/debug/templates.js, compile all the application code into
  // dist/debug/require.js, and then concatenate the require/define shim
  // almond.js and dist/debug/templates.js into the require.js file.
  grunt.registerTask("default", "concat min less mincss jst");
  grunt.loadNpmTasks('grunt-contrib');
};
@shama
Copy link
Member

@shama shama commented Jan 9, 2013

Thanks for the info. If you get a chance, could you try the latest grunt and grunt-contrib-watch? Let me know if it still runs up the CPU. We've made a lot of changes since v0.1.4 in the watch. I'll try your setup shortly to see if I can duplicate. Thanks!

@flynfish
Copy link
Author

@flynfish flynfish commented Jan 9, 2013

I updated to the lastest grunt and grunt-contrib versions and now the process never gets above 1% cpu, so I would say you guys did some awesome work on that update!

When running watch now and updating a less file, I get this error:

running "less:compile" (less) task
Warning: Object #<Object> has no method 'expandFiles' Use --force to continue.

Aborted due to warnings.

Any ideas on what I need to change to get it to work? (I already update the registerTask default alias to use an array instead of string"

@shama
Copy link
Member

@shama shama commented Jan 9, 2013

Awesome, glad to hear!

To get less working, you'll need to install the Grunt v0.4 version with: npm install git://github.com/gruntjs/grunt-contrib-less.git. In fact all of your plugins will need updating if moving to Grunt v0.4 (https://github.com/gruntjs/grunt/wiki/Upgrading-from-0.3-to-0.4). FWIW, I use Grunt v0.4 exclusively. Just know it is still under development so not all grunt plugins support it yet and things can change.

Another thing to try if staying on Grunt v0.3 for the time being. Try uninstalling grunt-contrib-watch and using the previously built-in watch task in Grunt v0.3. grunt-contrib-watch is intended as a replacement for the watch task in Grunt v0.4. We just had Grunt v0.3 support on the v0.1.x branch to get some early testers. :)

@flynfish
Copy link
Author

@flynfish flynfish commented Jan 9, 2013

I tried upgrading to grunt .4 and upgrading the grunt plugins but I ran into some issues I don't have time to figure out. I removed the bootstrap less files from being watched and that dropped the cpu to under 5%.

@shama
Copy link
Member

@shama shama commented Jan 28, 2013

Hopefully the other issues you got figured out :) Otherwise please reopen if one of those is with the watch task, thanks!

@shama shama closed this Jan 28, 2013
@alexbeletsky
Copy link

@alexbeletsky alexbeletsky commented Mar 23, 2013

I'm having the same issue on my Windows 8, cpu level near 70%, watching just a few files.

@Jesterovskiy
Copy link

@Jesterovskiy Jesterovskiy commented May 22, 2013

Have issue with high load too.

Versions:
"grunt": "~0.4.1",
"grunt-contrib-watch": "~0.4.3",
"grunt-contrib-sass": "~0.3.0"

System: Ubuntu 12.04 LTS

Gruntfile:

...
   watch: {
      sass: {
        files: ['app/stylesheets/{,**/}*.scss'],
        tasks: ['sass:dev']
      },
      html: {
        files: ['app/index.html'],
        tasks: ['html:dev']
      }
    },

  html: {
      dev: {
        options: {
          src: 'app/index.html',
          dest: 'temp/index.html',
          vars: config('development')
        }
      }
    },

    sass: {
      dev: {
        options: {
          style: 'expanded'
        },
        files: {
          'temp/stylesheets/application.css': 'app/stylesheets/application.scss'
        }
      }
    },
...
@shama
Copy link
Member

@shama shama commented May 22, 2013

Try running with --verbose to see which files it is watching. Also which version of node?

@Jesterovskiy
Copy link

@Jesterovskiy Jesterovskiy commented May 22, 2013

Task is watching 45 scss files and version of node is v0.10.4.

@shama
Copy link
Member

@shama shama commented May 22, 2013

I'm curious if the problem is within this lib or just watching in general (as that greatly differs on environments). Are you game to try some examples for me?

Could you first make sure any previous node processes are not running with killall -9 node and then run this example to see if that causes the CPU spike?

var fs = require('fs');
[
  // Please add a bunch more files
  'app/stylesheets/application.scss',
].forEach(function(file) {
  fs.watch(file, function(event, filename) {
    console.log(file + ' was ' + event);
  });
});
@Jesterovskiy
Copy link

@Jesterovskiy Jesterovskiy commented May 23, 2013

Not fully understand, how to run this example. Can you explain, please =)

@shama
Copy link
Member

@shama shama commented May 23, 2013

Oh that example will watch files using the core node.js api because I want to see if the problem is from this watch task or with any file watching.

Throw that example into a file named test.js. Add a bunch more actual file paths and then do this in your terminal: killall -9 node && node test.js to kill any other running node process and then run the example. Thanks!

@Jesterovskiy
Copy link

@Jesterovskiy Jesterovskiy commented May 23, 2013

Thank you for explain) I can't find other process of node on machine and have this error on start:

vagrant@precise32:/vagrant$ killall -9 node                
node: no process found
vagrant@precise32:/vagrant$ node test.js

fs.js:1051
    throw errnoException(process._errno, 'watch');
          ^
Error: watch ENOENT
    at errnoException (fs.js:1019:11)
    at FSWatcher.start (fs.js:1051:11)
    at Object.fs.watch (fs.js:1076:11)
    at /vagrant/test.js:31:6
    at Array.forEach (native)
    at Object.<anonymous> (/vagrant/test.js:30:3)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
@shama
Copy link
Member

@shama shama commented May 23, 2013

Thanks for giving it go. That error means one of the files you've tried doesn't exist. Could you try on only files that exists?

@Jesterovskiy
Copy link

@Jesterovskiy Jesterovskiy commented May 24, 2013

Thanks, you're right) It's my bad knowledge of english =(
This is results:
configuration

watch: {
      sass: {
        files: ['app/stylesheets/application.scss',
                'app/stylesheets/app/{,**/}*.scss',
                'app/stylesheets/lib/{,**/}*.scss'
               ],
        tasks: ['sass:dev']
      },
sass: {
      dev: {
        options: {
          style: 'expanded'
        },
        files: {
          'temp/stylesheets/application.css': 'app/stylesheets/application.scss'
        }
      },

screenshot of htop when run watch:sass
screen shot 2013-05-24 at 11 07 21 am

configuration

var fs = require('fs');
[
  // Please add a bunch more files
  'app/stylesheets/application.scss',
  'app/stylesheets/app/_authentication.scss',
  'app/stylesheets/app/_dynamic-content.scss',
  'app/stylesheets/app/_main.scss',
  'app/stylesheets/app/_news-feed.scss',
  'app/stylesheets/app/crime-reporting/_incident-show.scss',
  'app/stylesheets/app/crime-reporting/_incidents.scss',
  'app/stylesheets/app/crime-reporting/_print-content.scss',
  'app/stylesheets/app/crime-reporting/_subjects.scss',
  'app/stylesheets/app/crime-reporting/_video-edit.scss',
  'app/stylesheets/app/features/_comments.scss',
  'app/stylesheets/app/features/_entities.scss',
  'app/stylesheets/app/features/_groups.scss',
  'app/stylesheets/app/features/_messages.scss',
  'app/stylesheets/app/features/_premises.scss',
  'app/stylesheets/app/features/_statistics.scss',
  'app/stylesheets/app/features/_users.scss',
  'app/stylesheets/app/headers/_action-bar.scss',
  'app/stylesheets/app/headers/_header.scss',
  'app/stylesheets/app/plugins/_custom-redactor.scss',
  'app/stylesheets/app/plugins/_on-off-switch.scss',
  'app/stylesheets/app/sidebars/_left.scss',
  'app/stylesheets/app/sidebars/_right.scss',
  'app/stylesheets/app/ui/_flashes.scss',
  'app/stylesheets/app/ui/_forms.scss',
  'app/stylesheets/app/ui/_tables.scss',
  'app/stylesheets/lib/_lt-ie7.scss',
  'app/stylesheets/lib/_lt-ie8.scss',
  'app/stylesheets/lib/_lt-ie9.scss',
  'app/stylesheets/lib/_mixins.scss',
  'app/stylesheets/lib/bootstrap/_custom-icons.scss',
  'app/stylesheets/lib/bootstrap/_ie7buttonfix.scss',
  'app/stylesheets/lib/bootstrap/_ie8buttonfix.scss',
  'app/stylesheets/lib/bootstrap/_mixins-custom.scss',
  'app/stylesheets/lib/bootstrap/_modal.scss',
  'app/stylesheets/lib/bootstrap/_overrides.scss',
  'app/stylesheets/lib/bootstrap/_popover.scss'
].forEach(function(file) {
  fs.watch(file, function(event, filename) {
    console.log(file + ' was ' + event);
  });
});

screenshot of htop when run node test.js
screen shot 2013-05-24 at 11 08 51 am

@shama
Copy link
Member

@shama shama commented May 24, 2013

Hmm bummer. I'll reopen although I'm not sure how to fix as I can't reproduce and I am out of ideas. I really appreciate you trying those examples for me @Jesterovskiy! If someone has this issue and discovers a fix please let us know! Thanks.

@shama shama reopened this May 24, 2013
@alexbain
Copy link

@alexbain alexbain commented May 24, 2013

Just chiming in to say that we're experiencing the same issue watching around 100 files. We ran the same test above and didn't see any serious CPU usage, so no idea what's causing it.

@Jesterovskiy
Copy link

@Jesterovskiy Jesterovskiy commented May 27, 2013

Some test with https://github.com/shama/gaze shows same situation: 5 process and quiescent load 20-23% of cpu. I will dig deeper =)

@Jesterovskiy
Copy link

@Jesterovskiy Jesterovskiy commented May 27, 2013

I found the problem! =) In Gaze by default interval sets https://github.com/shama/gaze/blob/master/lib/gaze.js#L36 but in fs lib we see different default value https://github.com/joyent/node/blob/master/lib/fs.js#L1128 I changed this value in my test file and the result - no more high cpu load =)
configuration

var fs = require('fs'),
    opts = { persistent: true, interval: 5007 };
[
  // Please add a bunch more files
  'app/stylesheets/application.scss',
  'app/stylesheets/app/_authentication.scss',
  'app/stylesheets/app/_dynamic-content.scss',
  'app/stylesheets/app/_main.scss',
  'app/stylesheets/app/_news-feed.scss',
  'app/stylesheets/app/crime-reporting/_incident-show.scss',
  'app/stylesheets/app/crime-reporting/_incidents.scss',
  'app/stylesheets/app/crime-reporting/_print-content.scss',
  'app/stylesheets/app/crime-reporting/_subjects.scss',
  'app/stylesheets/app/crime-reporting/_video-edit.scss',
  'app/stylesheets/app/features/_comments.scss',
  'app/stylesheets/app/features/_entities.scss',
  'app/stylesheets/app/features/_groups.scss',
  'app/stylesheets/app/features/_messages.scss',
  'app/stylesheets/app/features/_premises.scss',
  'app/stylesheets/app/features/_statistics.scss',
  'app/stylesheets/app/features/_users.scss',
  'app/stylesheets/app/headers/_action-bar.scss',
  'app/stylesheets/app/headers/_header.scss',
  'app/stylesheets/app/plugins/_custom-redactor.scss',
  'app/stylesheets/app/plugins/_on-off-switch.scss',
  'app/stylesheets/app/sidebars/_left.scss',
  'app/stylesheets/app/sidebars/_right.scss',
  'app/stylesheets/app/ui/_flashes.scss',
  'app/stylesheets/app/ui/_forms.scss',
  'app/stylesheets/app/ui/_tables.scss',
  'app/stylesheets/lib/_lt-ie7.scss',
  'app/stylesheets/lib/_lt-ie8.scss',
  'app/stylesheets/lib/_lt-ie9.scss',
  'app/stylesheets/lib/_mixins.scss',
  'app/stylesheets/lib/bootstrap/_custom-icons.scss',
  'app/stylesheets/lib/bootstrap/_ie7buttonfix.scss',
  'app/stylesheets/lib/bootstrap/_ie8buttonfix.scss',
  'app/stylesheets/lib/bootstrap/_mixins-custom.scss',
  'app/stylesheets/lib/bootstrap/_modal.scss',
  'app/stylesheets/lib/bootstrap/_overrides.scss',
  'app/stylesheets/lib/bootstrap/_popover.scss'
].forEach(function(file) {
  fs.watchFile(file, opts, function(event, filename) {
    console.log(file + ' was ' + event);
  });
});
@shama
Copy link
Member

@shama shama commented May 27, 2013

@Jesterovskiy Awesome find! Thanks for looking into this! That makes a lot of sense. I'll have to increase the default interval for polling in gaze. In the meantime that setting is available through the options: options: { interval: 5007 }. Thanks! :D

@jrajav
Copy link

@jrajav jrajav commented Apr 21, 2014

I see that this has been closed but I still see grunt using about 9% (of a 2.4 i5) on a watch of a few dozen files on OS X. Doesn't seem like a lot but if I leave the watch running all day it keeps power management from kicking in and costs me battery life. I'll play around with that polling interval option, I just wanted to mention it here so that you know the defaults aren't ideal.

@shama
Copy link
Member

@shama shama commented Apr 22, 2014

FWIW, the next version of gaze uses native OS events for everything and will dramatically reduce the CPU usage. In the meantime, just make sure it's not matching unintended files grunt watch -v, set a higher interval.

@showrav017
Copy link

@showrav017 showrav017 commented Dec 10, 2014

Hi,

I know its a very old issue but i face this problem again on My windows 8.1 PC.

Its regarding the Sails.js mvc. I have tried all the above suggestions but these were not working for me. I have to commented out the codes under Create watcher per target line from the file tasks/watch.js of this module.

I just stop the watchers to pushing items on it under a infinite type loop. I just did it so don't know what happened further.

Hope it will help others temporarily.

Thanks

@joeyespo
Copy link

@joeyespo joeyespo commented Dec 14, 2014

Also seeing a CPU usage on a Windows 7 machine. And high memory usage (> 100MB).

@tigrr
Copy link

@tigrr tigrr commented Dec 20, 2014

Had the same problem.
It was caused by Grunt watching "node_modules" directory.
https://github.com/gruntjs/grunt-contrib-watch#why-is-the-watch-devouring-all-my-memorycpu

@timomayer
Copy link

@timomayer timomayer commented Feb 6, 2015

i have the same issue!

@pwdonald
Copy link

@pwdonald pwdonald commented Feb 16, 2015

I am also experiencing this issue. It starts out running at around 10-20%, then after about five minutes it shoots up to 99% CPU and stays there until I stop the watch task. I was unaware this was happening on my desktop but when I run it on my laptop it gets very warm and the fan spins constantly, which is what alerted me to the issue.

Edit: Just noticed the comment about watching the node_modules folder, that was my problem as well. Once I specified the watch directories to exclude that it fixed the issue.

@timomayer
Copy link

@timomayer timomayer commented Feb 16, 2015

@pwdonald exactly same for me. but i have node_modules excluded already...

@jifeon
Copy link

@jifeon jifeon commented Feb 25, 2015

Have the same problem on Yosemite on mac mini, 50% CPU for couple hundreds of scripts. I use the same configuration on fedora and grunt uses less than 1%

@jifeon
Copy link

@jifeon jifeon commented Feb 25, 2015

I've found out that updating gaze to 0.6.4 solves my issue. PR about it: #412

@jmorvan
Copy link

@jmorvan jmorvan commented Jun 16, 2015

@tikosar spot on! I was watching node_modules! tku

@Kungitoo
Copy link

@Kungitoo Kungitoo commented Sep 16, 2015

I've had the issue of watching node_modules/ as well. I've added

   '!node_modules/**/*.*'

like:

    watch: {
        options: {
            livereload: true
        },
        express: {
            files: ['**/*.js', '!node_modules/**/*.*'],

Should you maybe disable watching node_modules/ by default since I don't see why anyone would do that anyways and they could always enable it manually?

@yang
Copy link

@yang yang commented Sep 25, 2015

I'm seeing the same disparity in CPU usage between Linux (lower/more manageable) and Mac (high). I'm using 0.6.1.

@WayneKeenan
Copy link

@WayneKeenan WayneKeenan commented Mar 8, 2016

obviously ymmv, but just thought I mention another potential biggie that caused slowdown on my Mac which was solved by ignoring the bower_components folder, e.g. !app/bower_components/**/*.*

before : ~3100 files using ~%60 CPU usage
after: ~10 files using ~5% CPU usage

versions:

grunt-contrib-watch v0.6.1
grunt-cli v0.1.13
grunt v0.4.5
node v5.5.0

I would have thought watching 3100 files would not be such a problem giving the Mac's FSEvent API is supposed to be fast and light weight.

But, I've just noticed that the grunt-contrib-watch package.json doesn't explicitly mention the node module https://www.npmjs.com/package/fsevents as a dependancy and uses gaze which at first glance also doest use node's fsevent. (this was just a cursory check, but given the behaviour it would seem it doesn't)

I guess a potential fix (for very large projects) is to use fsevent on Mac (either directly in grunt-watch or by patching node's gaze)

@rbosneag
Copy link

@rbosneag rbosneag commented Mar 10, 2016

My special dumb case:

Took me 3h (of looking in watch source and gaze and resetting Gruntfile.js to a minimum and update packages one by one and changing versions, etc) to realize that my task was configured as follows:

grunt.registerTask('watch', ['watch']);

Guess what? Recursion! CPU went in 100% instantly, no info in terminal, verbose mode showed watch task running continuously.

That was stupid but made me laugh in the end.

I don't think my solution will help someone but I posted it just to cover all scenarios...

@saltcod
Copy link

@saltcod saltcod commented Jun 30, 2017

Someone had composer'd a /vendor/ dir in my theme. Ignoring that with '!/vendor/' did the trick for me. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.