New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

high cpu usage when running meteor from npm scripts block #4314

Closed
dferber90 opened this Issue May 2, 2015 · 24 comments

Comments

Projects
None yet
@dferber90
Contributor

dferber90 commented May 2, 2015

_9 Upvotes_ ## Summary
When running Meteor from npm's scripts-block a node process eats >100% CPU.

Details

There is a nice way of using npm's package.json file to save characters when starting Meteor with lots of options for development. Described in detail here,

Basically, Meteor can be started with $ npm start by including a file (called package.json) like the one below in the Meteor app.

{
  "name": "todos",
  "scripts": {
    "start": "meteor run"
  },
  "private": true
}

When running Meteor with $ npm start everything works as usual. But after letting it run for about 1,5minutes, the fans on my MacBook Pro go crazy. The cause of this is that a node process takes up more than 100% CPU and nearly 200M memory.

Reproduction

The reproduction repository contains the plain todos example. I only added a package.json file.

$ git clone git@github.com:dferber90/meteor-npm-issue.git
$ cd meteor-npm-issue
$ npm start

# now, view %CPU and MEM of a node process in a new terminal tab
$ top

When running the same app with the plain old $ meteor run command instead of $ npm start, everything works fine. I am not sure what is causing this.

@ekatek

This comment has been minimized.

Contributor

ekatek commented May 6, 2015

This is kind of cool -- I have never running meteor through npm before. Thanks for reporting! Tool performance is an ongoing project for us, so while we might not concentrate on debugging this explicitly in the near future (because running with meteor run works fine), it is great to keep it in mind.

@TimFletcher

This comment has been minimized.

TimFletcher commented Jan 11, 2016

I'm running into this issue too. My meteor app requires that I run a couple of other node processes at the same time so I just threw them all in as npm scripts so can just do npm start and run them all. Guess I'll have to find another way to run them all. Foreman, perhaps.

@ShockiTV

This comment has been minimized.

ShockiTV commented Jan 12, 2016

+1 seems we have to maintain bash script for now as shell aliases are not very git/team friendly

@Alino

This comment has been minimized.

Alino commented Jan 12, 2016

+1

simple meteor run is not enough for people who are doing more complicated projects where you need to start the whole environment with a single command

we need to start meteor with a ton of other scripts and commands bundled into one npm start script
This is really elementary thing to do, and it should be 100% supported, so please increase priority of this issue.

@ShockiTV

This comment has been minimized.

ShockiTV commented Jan 12, 2016

As mentioned in that linked Webpack meteor React topic, you can create it this way

My start.js ( we are running even DEV fully dockerized => need to configure like all ENV variables. I am pasting just MONGO_URL here as an example )

#!/usr/bin/env node

var spawn = require('child_process').spawn;
//var appDir = process.cwd();
var execEnv = process.env;
var envDup = {};
var someVar;

//process.chdir(appDir + '/app');

for (someVar in execEnv) {
    envDup[someVar] = execEnv[someVar];
}
envDup["MONGO_URL"] = 'YOUR_MONGO_URL';

var options = {
    env: envDup
};

var m = spawn('meteor', ['--settings', 'YOUR_SETTINGS_JSON', '--port', '3000'], options);

m.on('exit', function() {
    console.log('exiting as child died');
    process.exit();
});

m.stdout.on('data', function (data) {
    console.log("m:"+data);
});

process.on( "SIGINT", function() {
    m.kill("SIGINT");
    console.log('killing child');
} );

And than run it as npm run start
package.json:

{
  "name": "myApp",
  "scripts": {
    "start": "node start.js",
  }
}

Problem seems to be with the stdio pipes.
As soon as I switch to stdio: inherit in options, it eats 170% of my 4core and never drops.
Also you can notice that this working (no additional CPU consumption) stream listening way there is missing quite noticeable part of standard meteor run output normally visible if u run it directly.
You can add displaying errors using this style too:

child.stderr.on('data',
    function (data) {
        console.log('err data: ' + data);
    }
);

So maybe if Meteor just outputs on console better way, it can stop ruining CPU during standard usage.
Have fun

@Alino

This comment has been minimized.

Alino commented Jan 12, 2016

This brings unnecessary complexity, we've got a start shell script where we got already everything and now we have to run another start javascript which starts meteor.

@ShockiTV

This comment has been minimized.

ShockiTV commented Jan 12, 2016

I think the old ( using by Meteor internally ) and new ( installed by devs locally ) Node streams cooperation is kinda borked.
Simple nvm-ing to 0.10.40 node version bring both on same level and fixed CPU issues.
Have fun and thanks for all the fishes.

@ShockiTV

This comment has been minimized.

ShockiTV commented Jan 14, 2016

As we need to run also chimp and I would prefer still use current npm by default and not switch by hand so I prepared little wrapper for brew installed nvm on my OSX.

Roberts-MacBook-Pro:~ shock$ cat .bash_profile

export NVM_DIR=~/.nvm
. /usr/local/opt/nvm/nvm.sh
alias npm="~/.npm_wrapper.sh"

Roberts-MacBook-Pro:~ shock$ cat .npm_wrapper.sh

#!/bin/bash

readonly METEOR_NODE_VERSION="0.10.40"

# source nvm script installed by brew
. /usr/local/opt/nvm/nvm.sh

if [[ "$1" == "run" && ( "$2" == "start" || "$2" == "testEnv" ) ]]
then
  echo "Using npm for $METEOR_NODE_VERSION"
  nodePath=$(nvm which $METEOR_NODE_VERSION)
  npmPath=$(echo $nodePath | sed 's/node$/npm/')
  exec "$npmPath" $@
else
  # exec with current node
  nodePath=$(nvm which current)
  npmPath=$(echo $nodePath | sed 's/node$/npm/')
  exec "$npmPath" $@
fi

echo "Something went horribly wrong in npm_wrapper.sh"
exit 1

Bye

@camstuart

This comment has been minimized.

camstuart commented Feb 20, 2016

+1 Same issue here. My macbook fans go crazy when i run npm start I think having this as a standard way of launching meteor apps is ideal. No matter the project a developer should be able to clone, cd into the directory and run the app in a standard way. Then the projects package.json can control what happens next.

My only wish is that npm script accepted an array of commands like this:

"scripts": {
    "start": [
       "echo 'loading git packages'",
       "mgp"
       "echo 'launching application'",
       "meteor --settings settings-development.json"
   ]
  }

Instead of what I am currently doing:

"scripts": {
    "start": "echo 'loading git packages'; mgp; echo 'launching application'; meteor --settings settings-development.json"
  }

But otherwise, I'm all for this approach.

@svda

This comment has been minimized.

Contributor

svda commented Mar 6, 2016

+1

@zimt28

This comment has been minimized.

zimt28 commented Mar 8, 2016

+1. The current beta creates a package.js file with the start script, so this issue should be fixed before the 1.3 release.

@chmac

This comment has been minimized.

chmac commented Mar 11, 2016

We're seeing this on several of our team's and client's machines on OSX. Would be great to get a fix or a workaround...

@ghost

This comment has been minimized.

ghost commented Mar 14, 2016

+1

meteor: 1.3-beta.16
node: v4.4.0
npm: 2.14.20

@BlooJeans

This comment has been minimized.

BlooJeans commented Mar 18, 2016

+1
meteor: 1.3-beta.16
npm: 2.14.4
throwing my run command in a bash file fixed the issue

@nadeemja

This comment has been minimized.

nadeemja commented Mar 20, 2016

+1

1 similar comment
@st3phan

This comment has been minimized.

st3phan commented Mar 26, 2016

+1

@krstffr

This comment has been minimized.

krstffr commented Mar 30, 2016

This is still true post the 1.3 official release.

Also true if you run your tests using npm test (and make it run the new meteor test command).

@Alino

This comment has been minimized.

Alino commented Mar 30, 2016

Guys, if you are on Meteor 1.3
maybe you could try meteor npm start or something like that.
I have read some news that you can use npm in meteor cli now.

I haven't migrated to meteor 1.3 yet, so I have no experience with this, but it might solve the issue, if npm runs from meteor process. Please could someone confirm this?

@camstuart

This comment has been minimized.

camstuart commented Mar 30, 2016

After chatting to people on slack and in some meet up's, it seems the general opinion is that this is an issue with the very old node version that ships with Meteor. In the "What’s up next?" section of the 1.3 release announcement (http://hubs.ly/H02w6gg0), they mention node is getting updated next. So even though Meteor itself has been updated, node remains on the same version. So no improvement on this issue. I tried updating my node version only to realise that the node that underlies Meteor has nothing to do with my own version of node... oops.

@krstffr

This comment has been minimized.

krstffr commented Mar 30, 2016

@Alino even if that works it's still a workaround, and not the standard way of running things.

The point of using npm start is that everyone know how to do it, and it should not add a 170% increase in CPU load.

EDIT: It actually does solve it for me, and @camstuart is probably correct in that it's the old node version that needs an upgrade!

@Alino

This comment has been minimized.

Alino commented Mar 30, 2016

good to know that it works!
You are right that it is not standard way to run things but yet It is acceptable and simple workaround in my opinion. At least something. :)

@Sanjo

This comment has been minimized.

Contributor

Sanjo commented Apr 18, 2016

Using meteor npm start instead of npm start solves this issue for me.

I have the following versions installed:

  • Meteor: 1.3
  • Node.js: v4.4.1
  • NPM: 2.14.20
@jbaxleyiii

This comment has been minimized.

jbaxleyiii commented Jun 26, 2016

@benjamn do you have any idea why this is happening?

@hwillson

This comment has been minimized.

Member

hwillson commented Jan 30, 2017

This isn't happening when calling meteor npm ... (see #4314 (comment)), which is the recommended way of using the npm CLI with Meteor. Given this, and the age of this issue, I'll close it off. Happy to re-open if anyone notices this happening with a current version of Meteor. Thanks for posting this originally!

@hwillson hwillson closed this Jan 30, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment