Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: run node directly when streaming #10032

Merged
merged 1 commit into from Feb 13, 2019

Conversation

Projects
None yet
4 participants
@nolanlawson
Copy link
Collaborator

nolanlawson commented Feb 13, 2019

Based on some research I did in nolanlawson/pinafore#971 it seems that, somewhat surprisingly, running node directly instead of using npm run [script] consumes significantly less memory.

I'm testing this out in production right now, and for my server running malfunctioning.technology, freedom.horse, and toot.cafe, I previously saw these npm processes taking up 57728 kilobytes total of RSS (Resident Set Size), and all node/npm processes combined were taking 354168. After this change, that drops to 291588 and the npm processes are gone.

You can test this yourself by running something like:

ps aux | grep 'node\|npm'

You should see output like:

malftech 16962  0.0  0.2 1114420 18472 ?       Ssl  Feb06   0:00 npm
malftech 17021  0.0  0.0   4636   740 ?        S    Feb06   0:00 sh -c node ./streaming/index.js
malftech 17023  0.0  0.3 900956 31700 ?        Sl   Feb06   0:00 node ./streaming/index.js
malftech 17029  0.0  0.4 1199032 38392 ?       Sl   Feb06   0:05 /usr/bin/node /home/malftech/live/streaming/index.js

Directly running node should remove that npm process using 18472kB of RSS.

BTW I am not a Linux perf guru, so I am happy to have someone with more devops experience take a look at this first.

@@ -9,7 +9,7 @@ WorkingDirectory=/home/mastodon/live
Environment="NODE_ENV=production"
Environment="PORT=4000"
Environment="STREAMING_CLUSTER_NUM=1"
ExecStart=/usr/bin/npm run start
ExecStart=/usr/bin/node ./streaming

This comment has been minimized.

@nolanlawson

nolanlawson Feb 13, 2019

Author Collaborator

The ./index.js is implied

@hugomd

This comment has been minimized.

Copy link

hugomd commented Feb 13, 2019

I agree that it's better to call node directly, rather than npm start. However, do be wary that this means an pre or post scripts defined in package.json won't run 👍

@nolanlawson

This comment has been minimized.

Copy link
Collaborator Author

nolanlawson commented Feb 13, 2019

Good call, I don't see any prestart/poststart scripts in package.json 👍

@M-D-M

This comment has been minimized.

Copy link

M-D-M commented on 9d4371a Feb 13, 2019

Just tested this change on a vanilla 2.7.1 instance. No ill effects so far. Instance booted and running fine.

@nolanlawson

This comment has been minimized.

Copy link
Collaborator Author

nolanlawson commented Feb 13, 2019

So I just checked the documentation for ps, and apparently it lists RSS in kB, not bytes. So this change actually cut 57.7MB for me (on a system running three Mastodon servers, so ~18-20MB per server). I edited the top comment to reflect that.

@Gargron Gargron merged commit 658b462 into master Feb 13, 2019

11 checks passed

ci/circleci: build Your tests passed on CircleCI!
Details
ci/circleci: check-i18n Your tests passed on CircleCI!
Details
ci/circleci: install Your tests passed on CircleCI!
Details
ci/circleci: install-ruby2.4 Your tests passed on CircleCI!
Details
ci/circleci: install-ruby2.5 Your tests passed on CircleCI!
Details
ci/circleci: install-ruby2.6 Your tests passed on CircleCI!
Details
ci/circleci: test-ruby2.4 Your tests passed on CircleCI!
Details
ci/circleci: test-ruby2.5 Your tests passed on CircleCI!
Details
ci/circleci: test-ruby2.6 Your tests passed on CircleCI!
Details
ci/circleci: test-webui Your tests passed on CircleCI!
Details
codeclimate All good!
Details

@Gargron Gargron deleted the nolan/node-memory branch Feb 13, 2019

staticsafe added a commit to staticsafe/mastible that referenced this pull request Feb 17, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.