Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(core): use jemalloc as the memory allocator #3541

Merged
merged 5 commits into from Mar 2, 2020
Merged

perf(core): use jemalloc as the memory allocator #3541

merged 5 commits into from Mar 2, 2020

Conversation

alessiodf
Copy link
Contributor

@alessiodf alessiodf commented Feb 26, 2020

Summary

This PR replaces the native memory allocator with jemalloc in Core to reduce memory fragmentation and results in a smaller memory footprint as confirmed by multiple Core nodes using the native memory allocator (glibc) vs jemalloc, where in all cases the jemalloc nodes consistently used less RAM. Furthermore, results from a sync from 0 test (using the same hardware with 2GB physical RAM to eliminate variability due to different specifications) demonstrate that jemalloc is more performant and uses far less memory by the end:

native memory allocator (glibc): Completed in 14h 54m 14s using 1.3GB RAM 30 minutes after completion.
jemalloc: Completed in 13h 37m 56s using 314.5MB RAM 30 minutes after completion.

Therefore a sync from 0 using jemalloc completed 1h 16m 18s faster than glibc on the same hardware and, upon finishing, was using ~1017MB less RAM.

On a different (much higher powered) machine with 32GB RAM, I rolled back 300,000 blocks on devnet and re-synced using glibc and jemalloc:

native memory allocator (glibc): Completed in 7 minutes and 20 seconds using 3.5GB RAM (!) 10 minutes after completion.
jemalloc: Completed in 6 minutes and 50 seconds using 410.9MB RAM 10 minutes after completion.

In that instance, jemalloc completed 30 seconds faster than glibc and used a staggering ~3173MB less RAM.

This PR does not change the default system memory allocator to jemalloc as this may not be desirable and we should not make such system-wide changes. It just makes the Core process use jemalloc and leaves everything else untouched.

Special update instructions:
Users should manually run sudo apt-get install libjemalloc-dev (Debian derivatives) or sudo yum install libjemalloc-devel (RedHat derivatives) before updating since they don't re-run the Core installer (which will automatically install it for new installations).

Update with ark update --no-restart then pm2 delete all and restart the relay/forger/core process(es) via ark {relay|forger|core}:start. Merely executing ark {relay|forger|core}:restart or letting the updater restart the processes is insufficient as it will not inject the LD_PRELOAD variable, presumably due to pm2 caching.
Now refactored so the normal ark update procedure is sufficient to restart Core as long as libjemalloc has been installed as above.

To confirm Core is now using jemalloc, run cat /proc/`pgrep -f "@arkecosystem/core/bin/run"`/smaps | grep jemalloc. The output should show references to libjemalloc.so to confirm the process is running with jemalloc.

Checklist

  • Documentation (if necessary)
  • Tests (if necessary)
  • Ready to be merged

@ghost
Copy link

ghost commented Feb 26, 2020

Thanks for submitting this pull request! A maintainer will review this in the next few days and explicitly select labels so you know what's going on.

If no reviewer appears after a week, a reminder will be sent out.

@ghost ghost added Complexity: Low labels Feb 26, 2020
packages/core/bin/run Outdated Show resolved Hide resolved
@ghost
Copy link

ghost commented Mar 2, 2020

A collaborator has approved this PR. A maintainer will merge this PR shortly. If it shouldn't be merged yet, please leave a comment saying so and we'll wait.

Thank you for your contribution!

@ghost
Copy link

ghost commented Mar 2, 2020

A collaborator has approved this PR. A maintainer will merge this PR shortly. If it shouldn't be merged yet, please leave a comment saying so and we'll wait.

Thank you for your contribution!

@faustbrian faustbrian merged commit 71daf5f into ArkEcosystem:2.6 Mar 2, 2020
@ghost
Copy link

ghost commented Mar 2, 2020

Your pull request has been merged and marked as tier 1. It will earn you $200 USD.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants