Very high CPU usage on 1.3.5.1 when running meteor

[scharf]

On my mac, when I start meteor, node takes 175% CPU and my laptop heats up. The same happens for a coworker too.

We came to the conclusion, that it may have to do with automatic download of some meteor tools.

Update:

The issue appears to be that running Meteor projects that use a version of Meteor less than 1.4 with the 1.4 Meteor tool results in extremely high CPU use. This impacts most users as the 1.4 tool is automatically downloaded and run now that it's been released (regardless of whether users are ready to update to 1.4).

---

NOTE: There is a workaround for this issue, as outlined by @abernix : #7491 (comment)

SECOND NOTE: this is fixed by 1.4.0.1. You just need to ensure you have that version on your machine. The download should happen automatically, but if not (check by running meteor --version OUTSIDE of an app), grab it with meteor update outside of an app.

[scharf]

After I upgrade to meteor 1.4 the problem is gone. But I should be able to continue working with 1.3.5.1..

[benjamn]

And you can continue working with 1.3.5.1! Any app whose .meteor/release file has METEOR@1.3.5.1 in it will automatically use the older version of the tool. This is a special feature of Meteor that you won't find in many (any?) other framework.

[scharf]

...sure I can use 1.3.5.1, but meteor did something so that the meteor watch process kills my laptop (it's quite hot in my office and having a a high load all the time made my laptop stop working because of overheating).

But 1.4 seems pretty stable already (we usually wait for the x.x.1 release ;-)

[billyvg]

We are also having the same issues using release 1.3.5.1 - meteor dev server runs at 100% CPU with this release, but it seems to be fine at 1.4.

Tried reset and reinstall meteor to no avail.

[marcuscarr]

I'm running into this problem, too, on 1.2.1.

@scharf @billyvg - Are you using CollectionFS? Kadira showed a lot of Mongo activity, and db.system.profile shows a lot of queries to the cfs.<myFileCollection>.filerecords collections.

[scharf]

I tried another (much simpler) app that connects to the same database -- there I don't see the problem.

[marcuscarr]

I commented out all the CFS code and Node is still running like crazy. Guess that was a red herring.

[scharf]

I tried to debug/profile the meteor node process, but I got this:

Your Node version (v0.10.46) has a partial support of profiler.
The stack frames tree doesn't show all stack frames due to low sampling rate.
The profiling data is incomplete and may show misleading results.
Update Node to v0.11.13 or newer to get full support.

When I break the running process, I end up in this file: https://github.com/meteor/meteor/blob/master/tools/index.js but I have no idea that is relevant, because there is no real stacktrace....

[djules75]

I have the exact same problem with 1.3.5.1 and CPU usage is stuck at 175% on the node process. Problem is gone when updating to 1.4.

I'm getting the problem even with a simple barebones app created with meteor create.
Same problem with 1.3.2.4: meteor create --release=1.3.2.4 => 175% cpu usage.

This happens only in development, not in production. I'm on Os X El Capitan.

If there is anyway to help you debug the problem please let me know.

[marcuscarr]

Looks to me like the solution is for Meteor to revoke the update until they fix this. 😁

[pss4281]

same stuff here. Having 1.2.1 app and 1.4 installed in the system makes 1.2.1 app eat 178% of my cpu (mb pro, el capitan). But if I only install 1.2.1 like this (curl https://install.meteor.com/?release=1.2.1 | sh) than it works fine. This is a production running app and upgrading to 1.4 isn't on our agenda (at least not now) so this is pretty annoying.

[pss4281]

It's getting even worse because meteor somehow manages to silently download 1.4 in the background and the whole story begins over again:

loop_forever {
  removing meteor;
  installing 1.2.1;
  hoping that it won't install 1.4 behind the scenes again 
}

😨

[scharf]

I have had this before: I really wonder why meteor changes the existing environment when a new version comes out. I would expect that the silent download of a new version does not influence the existing version.

[apendua]

Is it possible that this is a Node 4.x issue?

[berkaytheunicorn]

+1 mine is gone too after update to 1.4. it was happening all of my meteor apps with 1.3.2.4 and it just start today. yesterday same app was working fine.

[scharf]

I think the problem is, that suddenly meteor 1.3.5.1 uses node v4.4.7:

$ meteor --version
Meteor 1.3.5.1
$ meteor node --version
v4.4.7

My guess is that this may be the cause of the problem.

[pss4281]

@scharf could be the case. This is definitely not something app-specific since I see the same behaviour on a new empty 1.2.1 app

[abernix]

I am able to reproduce this problem, but have no solution at the moment.

@scharf What directory are you running that command in? It will behave different when run within a project directory versus outside a project directory but it's strange that your versions are not properly matched in that output.

Meteor 1.4 (the tool) should still be running 1.3.5.1 apps using the 1.3.5.1 dev_bundle on Node 0.10.x.

[scharf]

I does behave differently: when I am outside the project, I get

$ meteor --version
Meteor 1.4
$ meteor node --version
v4.4.7

inside the project, the meteor version is correct, but the node version is wrong:

$ meteor --version
Meteor 1.3.5.1
$ meteor node --version
v4.4.7

[abernix]

@scharf Interesting. I'm still able to reproduce the problem but my project is definitely using the correct node. Ok – questions!

While your project is running – Can you provide the output of following commands when executed from another shell in your project root?:

• echo process.version | meteor shell
• cat .meteor/release
• ls -la .meteor/local/dev_bundle .meteor/dev_bundle ~/.meteor/meteor ~/.meteor/packages/meteor-tool

[pss4281]

@abernix

• meteor shell hangs on any operation so can't really tell the output of this echo process.version | meteor shell
  
• cat .meteor/release => METEOR@1.2.1

➜  foo-bar-app git:(master) ls -la .meteor/local/dev_bundle .meteor/dev_bundle ~/.meteor/meteor ~/.meteor/packages/meteor-tool
ls: .meteor/dev_bundle: No such file or directory
ls: .meteor/local/dev_bundle: No such file or directory
lrwxr-xr-x  1 paul  staff  50 Jul 27 14:10 /Users/paul/.meteor/meteor -> packages/meteor-tool/1.4.0/mt-os.osx.x86_64/meteor

/Users/paul/.meteor/packages/meteor-tool:
total 16
drwxr-xr-x    6 paul  staff   204 Jul 27 14:09 .
drwxr-xr-x  181 paul  staff  6154 Jul 27 14:07 ..
drwxr-xr-x   15 paul  staff   510 Oct 27  2015 .1.1.10.1b51q9m++os.osx.x86_64+web.browser+web.cordova
drwxr-xr-x   15 paul  staff   510 Jul 27 14:09 .1.4.0.1vopth2++os.osx.x86_64+web.browser+web.cordova
lrwxr-xr-x    1 paul  staff    54 Oct 27  2015 1.1.10 -> .1.1.10.1b51q9m++os.osx.x86_64+web.browser+web.cordova
lrwxr-xr-x    1 paul  staff    53 Jul 27 14:09 1.4.0 -> .1.4.0.1vopth2++os.osx.x86_64+web.browser+web.cordova

[apendua]

@abernix For what it's worth, I've had similar problems with node process eating CPU like crazy when I was trying to pipe stdout from meteor to the parent process, i.e. meteor was spawned as a child process.

[acemtp]

Same here, I can run different meteor project (1.3 and less) and they all take 180% cpu on my mac.

[myktra]

@scharf I am experiencing all of this as well on a 1.3.5.1 project after the 1.4 tool download. However, my setup still reports Node v0.10.46 as the node version "in effect" inside my Meteor project:

$ meteor --version
Meteor 1.3.5.1                                
$ meteor node --version
v0.10.46

[tmeasday]

Fixed in 1.4.0.1, although I think I'll leave it open until node 4.5.0 is out, and we can revert the change.

[tmeasday]

Thanks again to @apendua and @abernix for the assist on this one.

[abernix]

@scharf As my last opinion on this within this issue (happy to discus further in the forums): I've been using Meteor for years now and, like Tom said, I don't believe what you said above has ever been true. Yes, meteor node and meteor npm were sometimes using the wrong version, but this has been fixed recently – even those commands themselves are very new, they were created in order to get ready for Meteor 1.4! In general, Meteor has done a great job switching between versions.

If the script you provided and the work arounds you're using to run exact Meteor versions are working for you, great! However this is exactly what the Meteor tool is supposed to do for you and I don't think it's expected that the user should have to work around it – meteor-tool should be quick, efficient and reliable at making this decision. That's the point. Can it improve? I'm sure. But work-around shell scripts with that sort of functionality, while great for temporary fixes, don't do anything to help the greater Meteor community, and don't work at all for the very large number of Windows users who may have similar issues. I hope nobody has to leave your script in place!

Your original suggestion to move all functionality into a shell script doesn't quite work as there are package resolution/downloading tasks that the Meteor code must handle. Aside from this particular issue as a hiccup (in an otherwise very very large upgrade), I'd encourage you to have trust in the Meteor tool to do the right thing – and if you experience problems, open issues with improvement ideas that will hopefully turn into PRs and improvements for everyone. In recent activity, I think preventing automatic downloads (#7445), intention of making this easier to find in prereleases, and on-going discussion in #7026 - the tool will improve further.

That being said. Everyone should update to 1.4.0.1 where this issue is hopefully settled! :)

meteor update --release 1.4.0.1

[scharf]

1.4.0.1 fixes the CPU problem! 👍

@abernix I started using meteor METEOR@1.3-beta.12 and I have seen a few surprising changes in node and npm versions without upgrading to the next version. We have always updated relatively quickly to the next version of meteor, but I think it is not ok to change the version under the hood while using an old version.

If the goal of meteor it to keep versions stable and this was just a bug, I am absolutely willing to trust meteor and drop my script (one thing less to worry about).

I have seen too many changing environments in the last 40 years. Therefore I try to protect my systems form outside influences as much as possible. Upgrades are absolutely necessary but I want to control when I do the upgrade and I want a way back it the new version causes problems.

One of the simplest solution is to have independent installations and set $PATH depending on the used version (like python virtualenv and nvm). A bit more complex is to have wrappers (scripts or in some other way) for each command that make inside a decision which version of the program to execute.

However: even with 1.4.0.1 the following happens:

$ meteor --version
Meteor 1.3.2.4
$ meteor node --version
v4.4.7
$ meteor npm --version
3.10.5

UPDATE: for some reasons it now works:

$ meteor --version
Meteor 1.3.2.4
$ meteor node --version
v0.10.43
$ node --version
v0.10.41

END UPDATE

This is exactly what I want to avoid. Or does meteor 1.3.2.4 work with node 4.4.7????

Fortunately, meteor is using the correct version of node, just meteor node is using the newest one.

OTOH: meteor npm install is using the wrong version of node and npm which can be fatal:

/Users/scharf/.meteor/packages/meteor-tool/.1.4.0-1.1rhgg5f++os.osx.x86_64+web.browser+web.cordova/mt-os.osx.x86_64/dev_bundle/bin/node /Users/scharf/.meteor/packages/meteor-tool/.1.4.0-1.1rhgg5f++os.osx.x86_64+web.browser+web.cordova/mt-os.osx.x86_64/tools/index.js npm install

[skishore]

I saw the same issue with an app on Meteor 0.9.1. Updating my meteor-tool version to 1.4.01 didn't fix it for me, though. One strange thing I noticed is that when run meteor, it selects a strange tool version:

.1.0.26.1jddtpq++os.osx.x86_64+web.browser+web.cordova

Here's what the diagnostics say:

$ meteor --version
Meteor 0.9.0.1
$ meteor node --version
v4.4.7
$ meteor npm --version
3.10.5

I tried to run meteor update, but got an error installing bcrypt: http://pastebin.com/2JdFkaSj

node-gyp is installed:

node-gyp@3.4.0  /usr/local/n/versions/node/5.2.0/lib/node_modules/node-gyp
node@6.2.2

I have no clue where I can go from here. Is there any way to get this old app working again?

[skishore]

My update is failing because of this error when adding the bcrypt package:

ld: library not found for -lgcc_s.10.5

In turn, that error is caused because I don't have Xcode 7. Macs are truly a hostile ecosystem for anyone who doesn't stay up to date... However, there's still the CPU issue with the old version of Meteor. Are releases prior to 1.0 still supported?

[c9s]

I understand that meteor wants the users to get the upgrade as soon as possible.

However, there are so many bugs in the new releases....
If you can't 100% guarantee that there is no new critical bugs in the latest release, or you should let the meteor users be able to keep all packages & tools on the previous stable version.

thus, I would like to suggest just to remove the auto-upgrade feature since it's breaking things when everytime a new release comes out.

[tmeasday]

@skishore - wow, still running an app on 0.9.1? I didn't know anyone was still doing that :). I think I can understand why the CPU issue might remain for you then. This problem will be fixed when node 4.5 is released (any minute now...). I would encourage you to update anyway, although it might be tricky to go straight from 0.9.1 to 1.4 in one go! (Sounds like you just need a build toolchain though).

@c9s - the problem with this kind of bug (where the recommended release causes a problem with running other releases) is it's kind of by definition impossible to discover until we recommend the release. However, to my knowledge this is the first time such an issue has happened (where the latest release causes issues to people who aren't running that release).

We don't anticipate bumping major versions of Node all that often, however :)

[mitar]

I run 0.8.3. :-)

[skishore]

@tmeasday thanks for the tips! In looking around, I wasn't able to find a way to install a sufficiently new xcode-select toolchain without installing Xcode 7, so I ended up doing that. Updating to meteor-tool 1.3 would probably also have worked for me.

My main project is up-to-date, but the reason I had one back at 0.9.1 was because it was a side project that I haven't touched in a year. I would be nice if there was testing that guaranteed that such projects would run without continual maintenance.

[tmeasday]

In case people are still seeing this because the tool is not autoupdating to 1.4.0.1 I added a note to the top of the issue

[tmeasday]

Closing this issue as it's fixed by 1.4.0.1. Hopefully anyone stung will find it if they google it.

[skishore]

I think Meteor 1.4.1 may have reintroduced this bug for apps still using older versions. I had somewhere between 100-200% CPU usage for a 1.4.0.1 app when the tool was updated, with no clients connected. The issue was immediately resolved upon updating to 1.4.1.

What kind of testing is there for apps running old versions of Meteor? The backwards compatibility feature is not useful if performance is affected so drastically.

[skishore]

Interestingly, now that I've downloaded meteor-tool@1.4.1, I can revert my app back to a 1.4.0.1 commit and everything works fine as well.

[tmeasday]

Hi @skishore, not seeing this. Are you able to reproduce at all?

[skishore]

Unfortunately I am completely unable to reproduce the issue. I tried creating a new app at release 1.4.0.1 by directly invoking meteor-tool@1.4.0.1 (i.e., to mimic the old setup that was failing) but that still worked well. Is it possible that the CPU usage was due to downloading the tool in the background? I've saw on another issue that the "There is a new release of meteor ready to download." message is only shown after the download is complete.

[tmeasday]

It sounds conceivable; or certainly unpacking the download perhaps.

[c9s]

Running meteor 1.4.1.2 with app (1.4.1.2) still causes very high CPU usage.

[abernix]

@c9s That doesn't match the original issue here which only occurred when springboarding a Meteor 1.4 app to a 1.3 (Node 0.10) app. It was actually an upstream Node issue and is definitely fixed. I cannot reproduce this problem you are referring to so if you're experiencing this (1.4.1.2 release and 1.4.1.2 app), please open a new issue with a fresh reproduction.