Set Java heap size lower than default

[jkutner]

The addition of the JAVA_OPTS in this PR will prevent R14 and R10 errors I've seen. The default JVM heap size is too high for Heroku's 512mb RAM on a free dyno. But the app seems to work fine with this heap setting.

Also, I gave the source project a try:

$ git clone https://github.com/metabase/metabase
$ cd metabase
$ heroku create
$ git push heroku master

And it worked on the first attempt. So I'm curious why the Heroku Button uses this repo instead? Is just because it's faster?

Thanks!

[agilliland]

Cool, thanks for the contribution @jkutner !

I'll have to mull this over a little bit because the down side to this is that you are forcing all launches of the app (via this script) to have those constrained memory settings. Ideally this is something that should come from the environment somehow so that it's configurable.

In the case of Heroku specifically, we have a separate repository for the Heroku Button just to make things a little faster and to make it easy to pin to our production releases.

I'm not as familiar with Heroku as @tlrobinson , but is there no way to set environment variables when launching dynos? Is it possible to include env settings in the one-click deploy recipes?

I think that ideally we'd have a default setting which uses nice tidy memory settings like are proposed here, but it should be easy to modify those settings if someone wants to upgrade to a larger dyno and has more resources available.

[jkutner]

What you want makes perfect sense, and is easy to do -- i just didn't want to intrude to much.

The first option is to replace the null-buildpack with the https://github.com/heroku/heroku-buildpack-jvm-common as the buildpack for the button. This will setup intelligent defaults for your app, and install JDK 8 (instead of using JDK 7, which is the default on the cedar stack image).

The only problem is that even with the intelligent defaults, I've see this app go over 512mb on a few of my tests (because -Xmx384m + Metaspace + other non-heap was just a little bit too much).

So you could just set JAVA_OPTS dynamically in your bin/start script the same way that Heroku does it with a chunk of code like this:

limit=$(ulimit -u)
case $limit in
512)   # 2X Dyno
  default_java_mem_opts="-Xmx768m"
  ;;
16384) # IX Dyno
  default_java_mem_opts="-Xmx2g"
  ;;
32768) # PX Dyno
  default_java_mem_opts="-Xmx12g"
  ;;
*) #1X Dyno
  default_java_mem_opts="-Xmx300m -Xss512k"
  ;;
esac

export JAVA_OPTS="$default_java_mem_opts -Dfile.encoding=UTF-8"

But I maintain the JVM buildpack, and I've recently decided to scale down the default 384mb for heap on the 1X, because I do see a lot of apps going over.
heroku/heroku-buildpack-jvm-common@aa94d2e

So maybe give the https://github.com/heroku/heroku-buildpack-jvm-common in app.json a try and see how things go.

After the black friday season, I will probably bring this down some more, and your app should be fine then.

Let me know if any of this needs clarification.

[wvengen]

After a little experimentation with heap values on 512MB Heroku dynos (free), I settled on 250MB.

[jkutner]

I've updated this PR so that it sets the heap size to 256m, but only for Free, Hobby and 1X dynos that have 512mb of RAM.

@agilliland it definitely seems that the experience is a little rough without this. And it seems fine with it. If the user scales the app up, the Xmx256 will be ignored, and a larger heap size will be used.

[ArtursO]

Hello,

I'm getting the R14 error on Heroku as well and I was wondering if this PR will fix it (once / if it's merged with master)? @wvengen @agilliland

2017-03-23T14:19:22.039165+00:00 heroku[web.1]: Error R14 (Memory quota exceeded) 2017-03-23T14:19:44.365455+00:00 heroku[web.1]: Process running mem=655M(128.0%) 2017-03-23T14:19:44.365539+00:00 heroku[web.1]: Error R14 (Memory quota exceeded) 2017-03-23T14:20:06.732959+00:00 heroku[web.1]: Process running mem=655M(128.0%) 2017-03-23T14:20:06.732959+00:00 heroku[web.1]: Error R14 (Memory quota exceeded) 2017-03-23T14:20:28.263892+00:00 heroku[web.1]: Process running mem=655M(128.1%) 2017-03-23T14:20:28.263892+00:00 heroku[web.1]: Error R14 (Memory quota exceeded) 2017-03-23T14:20:51.626301+00:00 heroku[web.1]: Process running mem=655M(128.1%) 2017-03-23T14:20:51.626473+00:00 heroku[web.1]: Error R14 (Memory quota exceeded) 2017-03-23T14:21:12.814718+00:00 heroku[web.1]: Process running mem=655M(128.1%) 2017-03-23T14:21:12.814718+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)

Using Hobby Basic (the $9/mo tier) for database and one free dyno.
version 0.23.0

[jkutner]

@ArtursO yes, I would expect this PR to fix that problem. But you can also override the heap settings yourself by running:

$ heroku config:set JAVA_TOOL_OPTIONS="-Xmx256m" -a <appname>

(Where <appname> is the name of your Heroku app)

[ArtursO]

@jkutner looks like I'm still getting the R14 memory error :(

2017-03-23T19:52:03.304461+00:00 app[web.1]: 03-23 19:52:03 �[1mWARN domassign.DeclarationTransformer�[0m :: Unable to find method for property speak-numeral. 2017-03-23T19:52:03.304089+00:00 app[web.1]: 03-23 19:52:03 �[1mWARN domassign.DeclarationTransformer�[0m :: Unable to find method for property cue. 2017-03-23T19:52:03.304827+00:00 app[web.1]: 03-23 19:52:03 �[1mWARN domassign.DeclarationTransformer�[0m :: Unable to find method for property cue-after. 2017-03-23T19:52:03.305002+00:00 app[web.1]: 03-23 19:52:03 �[1mWARN domassign.DeclarationTransformer�[0m :: Unable to find method for property pause-before. 2017-03-23T19:52:03.305147+00:00 app[web.1]: 03-23 19:52:03 �[1mWARN domassign.DeclarationTransformer�[0m :: Unable to find method for property play-during. 2017-03-23T19:52:03.305293+00:00 app[web.1]: 03-23 19:52:03 �[1mWARN domassign.DeclarationTransformer�[0m :: Unable to find method for property voice-family. 2017-03-23T19:52:03.305451+00:00 app[web.1]: 03-23 19:52:03 �[1mWARN domassign.DeclarationTransformer�[0m :: Unable to find method for property speak. 2017-03-23T19:52:03.305584+00:00 app[web.1]: 03-23 19:52:03 �[1mWARN domassign.DeclarationTransformer�[0m :: Unable to find method for property elevation. 2017-03-23T19:52:03.305714+00:00 app[web.1]: 03-23 19:52:03 �[1mWARN domassign.DeclarationTransformer�[0m :: Unable to find method for property stress. 2017-03-23T19:52:03.305867+00:00 app[web.1]: 03-23 19:52:03 �[1mWARN domassign.DeclarationTransformer�[0m :: Unable to find method for property speak-header. 2017-03-23T19:52:03.306085+00:00 app[web.1]: 03-23 19:52:03 �[1mWARN domassign.DeclarationTransformer�[0m :: Unable to find method for property pitch-range. 2017-03-23T19:52:18.768963+00:00 heroku[web.1]: Process running mem=618M(120.7%) 2017-03-23T19:52:18.769080+00:00 heroku[web.1]: Error R14 (Memory quota exceeded) 2017-03-23T19:52:39.916293+00:00 heroku[web.1]: Process running mem=618M(120.7%) 2017-03-23T19:52:39.916396+00:00 heroku[web.1]: Error R14 (Memory quota exceeded) 2017-03-23T19:53:02.136824+00:00 heroku[web.1]: Process running mem=618M(120.7%) 2017-03-23T19:53:02.136824+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)

[ArtursO]

I upgraded to one Hobby Dyno - This is what it's looking right now in terms of memory usage

[jkutner]

@ArtursO a Hobby dyno has exactly the same resources (memory) as a free dyno. The only difference is that it doesn't go to sleep when not in use. You may need to step up to Standard-2X, which has more RAM. At some point, with real usage, 512mb will probably not be enough for Metabase.

[salsakran]

Hey @jkutner -

Thanks so much for this PR and for persisting. We've been a bit overwhelmed and a lot of PRs have slipped through the cracks. The ulimit finger prints for the different dynos was new to me.

[ArtursO]

@jkutner thanks for letting me know and your help. Really appreciate it :)!