Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate and reduce memory usage #216

Closed
lihaoyi opened this Issue Sep 8, 2015 · 4 comments

Comments

Projects
None yet
1 participant
@lihaoyi
Copy link
Owner

lihaoyi commented Sep 8, 2015

Ammonite uses about 3x the memory usage of the default Scala REPL: starting at 400mb instead of 150mb, and growing significantly faster as well to >700mb while the normal REPL grows to >250mb as commands are entered

It's expected to take more memory, but 3x is a bit too much. There are probably low-hanging fruit to pick in bringing that number down

@lihaoyi

This comment has been minimized.

Copy link
Owner Author

lihaoyi commented Sep 17, 2015

Turns out that just adding a memory limit is enough to stop a greedy JVM using more and more RAM:

Before:
screenshot 2015-09-16 19 10 32

After -Xmx512M:
screenshot 2015-09-16 19 15 31

We could probably set the limit much lower, (200mb?) as long as there's an "easy" way to bump up the number when necessary

@lihaoyi

This comment has been minimized.

Copy link
Owner Author

lihaoyi commented Sep 17, 2015

Using the new G1 collector keeps the memory usage under control while still scaling without an upper limit where necessary:

Before:
screenshot 2015-09-16 19 52 47

After -XX:+UseG1GC:
screenshot 2015-09-16 19 51 31

This keeps the "total" memory overhead around 400+ megabytes, with basically no unnecessary growth. That may be enough for now

@lihaoyi

This comment has been minimized.

Copy link
Owner Author

lihaoyi commented Sep 17, 2015

And here's the result of the default collector, using System.gc() after every command:

screenshot 2015-09-16 20 00 51

There's a significant almost 2x slowdown over the earlier trials

The test input for all these trials is as follows

123
123
12
312
3
123
123
12
31
23
123
123
123
12
31
23
123
123
123
213
123
123
123
12
31
231
23
123
123
123
123
12
31
23
123
123
123
123
12
3
Array.fill(4000000)(100000)
123
123
12
312
3
123
123
12
31
23
123
123
123
12
31
23
123
123
123
213
123
123
123
12
31
231
23
123
123
123
123
12
31
23
123
123
123
123
12
3
Array.fill(4000000)(100000)
123
123
12
312
3
123
123
12
31
23
123
123
123
12
31
23
123
123
123
213
123
123
123
12
31
231
23
123
123
123
123
12
31
23
123
123
123
123
12
3
Array.fill(4000000)(100000)
123
123
12
312
3
123
123
12
31
23
123
123
123
12
31
23
123
123
123
213
123
123
123
12
31
231
23
123
123
123
123
12
31
23
123
123
123
123
12
3

@lihaoyi

This comment has been minimized.

Copy link
Owner Author

lihaoyi commented Sep 17, 2015

Closing this as "good enough" with 20138f5. Looks like we're looking at maybe a 20% heap-size-overhead and 40% OS-footprint-overhead over the default Scala REPL:

screenshot 2015-09-16 21 47 17

Which is probably good enough for now. The in-build Scala REPL processes the inputs about 3x as fast as Ammonite does, so there's clearly perf work to be done, but the memory consumption should be good enough for now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.