Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate and reduce memory usage #216

Closed
lihaoyi opened this issue Sep 8, 2015 · 4 comments
Closed

Investigate and reduce memory usage #216

lihaoyi opened this issue Sep 8, 2015 · 4 comments

Comments

@lihaoyi
Copy link
Member

@lihaoyi lihaoyi commented Sep 8, 2015

Ammonite uses about 3x the memory usage of the default Scala REPL: starting at 400mb instead of 150mb, and growing significantly faster as well to >700mb while the normal REPL grows to >250mb as commands are entered

It's expected to take more memory, but 3x is a bit too much. There are probably low-hanging fruit to pick in bringing that number down

@lihaoyi
Copy link
Member Author

@lihaoyi lihaoyi commented Sep 17, 2015

Turns out that just adding a memory limit is enough to stop a greedy JVM using more and more RAM:

Before:
screenshot 2015-09-16 19 10 32

After -Xmx512M:
screenshot 2015-09-16 19 15 31

We could probably set the limit much lower, (200mb?) as long as there's an "easy" way to bump up the number when necessary

Loading

@lihaoyi
Copy link
Member Author

@lihaoyi lihaoyi commented Sep 17, 2015

Using the new G1 collector keeps the memory usage under control while still scaling without an upper limit where necessary:

Before:
screenshot 2015-09-16 19 52 47

After -XX:+UseG1GC:
screenshot 2015-09-16 19 51 31

This keeps the "total" memory overhead around 400+ megabytes, with basically no unnecessary growth. That may be enough for now

Loading

@lihaoyi
Copy link
Member Author

@lihaoyi lihaoyi commented Sep 17, 2015

And here's the result of the default collector, using System.gc() after every command:

screenshot 2015-09-16 20 00 51

There's a significant almost 2x slowdown over the earlier trials

The test input for all these trials is as follows

123
123
12
312
3
123
123
12
31
23
123
123
123
12
31
23
123
123
123
213
123
123
123
12
31
231
23
123
123
123
123
12
31
23
123
123
123
123
12
3
Array.fill(4000000)(100000)
123
123
12
312
3
123
123
12
31
23
123
123
123
12
31
23
123
123
123
213
123
123
123
12
31
231
23
123
123
123
123
12
31
23
123
123
123
123
12
3
Array.fill(4000000)(100000)
123
123
12
312
3
123
123
12
31
23
123
123
123
12
31
23
123
123
123
213
123
123
123
12
31
231
23
123
123
123
123
12
31
23
123
123
123
123
12
3
Array.fill(4000000)(100000)
123
123
12
312
3
123
123
12
31
23
123
123
123
12
31
23
123
123
123
213
123
123
123
12
31
231
23
123
123
123
123
12
31
23
123
123
123
123
12
3

Loading

@lihaoyi
Copy link
Member Author

@lihaoyi lihaoyi commented Sep 17, 2015

Closing this as "good enough" with 20138f5. Looks like we're looking at maybe a 20% heap-size-overhead and 40% OS-footprint-overhead over the default Scala REPL:

screenshot 2015-09-16 21 47 17

Which is probably good enough for now. The in-build Scala REPL processes the inputs about 3x as fast as Ammonite does, so there's clearly perf work to be done, but the memory consumption should be good enough for now

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant