How to configure Java heap

[step76]

Is there a way to configure Java heap, or how to pass custom flags to the java VM?

[md5]

@step76 Right now, the only way to do this is to replicate the default command with your additional flags.

Here's what it looks like in a Dockerfile:

FROM jetty:9.3
CMD ["java","-Djava.io.tmpdir=/tmp/jetty","-Xmx1g","-jar","/usr/local/jetty/start.jar"]

Here's what it looks like on the command line:

$ docker run -d -p 80:8080 -p 443:8443 jetty java -Djava.io.tmpdir=/tmp/jetty -Xmx1g /usr/local/share/jetty/start.jar

[md5]

@gregw This seems to be a drawback of moving away from jetty.sh. Do you guys have any plans to introduce a jetty shell script that has less init cruft than jetty.sh but still allows for setting things like this via JAVA_OPTIONS?

[gregw]

We could look at that, but it would be very difficult to come up with a one size fits all script.

Isn't the solution here to modify the docker-entrypoint.sh script to inject $JAVA_OPTIONS into the default command line? Perhaps we change the default CMD to just be the arguments to java, as the entry point already handles that and injects a java command. That way if somebody explicitly specified a java command they could set whatever options they like... if they went with the default then they would be able to use JAVA_OPTIONS to control the the JVM.

I'd much rather just have a single script rather than a script that invokes another script and then jetty.

[md5]

@gregw If we put that into the entrypoint, then the CMD stops being self-documenting. Right now, the entrypoint only injects the default java command if a non-executable argument is passed as the first argument to the docker run command.

I'd rather not have the default CMD just be the arguments to java, since the user would still have to remember to add -jar /usr/local/share/jetty/start.jar in that case if they wanted to modify anything else, since the filename after -jar signals the end of JVM arguments and the start of main arguments.

I suppose that modifying the entrypoint to know about JAVA_OPTIONS is the least bad option. Another options could be to look for arguments starting with -X and -D in "$@", but there's also -javaagent and possibly other things that need to go before -jar.

Incidentally, I noticed while investigating this that the argument order in the entrypoint is weird. The -Dio.tmpdir is between -jar and its argument. It's been that way since #17. I wonder if that functionality even works...

[gregw]

@md5 quick explanation of the ordering and then I'll consider other points in another comment. The start.jar mechanism can look at the modules and determine that jvm args are needed (Eg if ALPN was enabled), so in that case start.jar will fork another JVM instance. Putting the -D after the -jar means that the JVM arg is for the JVM that will run jetty and not for the instance that will run start.jar. So that's how it works.

However, it should be safe to put it before the -jar as such values should be copied over to the target JVM if one is needed. Putting it after the -jar means that a second JVM is always needed. So let's open another issue to consider if that is the ordering we want: #38

[gregw]

So how about making the CMD:

CMD ["java","\$JAVA_OPTIONS,"-Djava.io.tmpdir=/tmp/jetty","-jar","/usr/local/jetty/start.jar"]

Note the $, so the entry point script can look for unexpanded $JAVA_OPTIONS and do the expansion. This would allows users to over-ride the command and decide to either use that expansion or instead directly specify all their args

[gregw]

@md5 Ah regarding the ordering issue - no that does indeed look broken! I'll comment on #38

[md5]

@gregw I'm not too keen on detecting the unexpanded $JAVA_OPTIONS... I started to write up something that looks if "$1" is java and $JAVA_OPTIONS is set to a non-empty value, but what I was writing would fall down if someone set both $JAVA_OPTIONS and a custom command starting with java.

@tianon Do you have any thoughts on this? To summarize:

• Our current CMD is CMD ["java","-Djava.io.tmpdir=/tmp/jetty","-jar","/usr/local/jetty/start.jar"]
• We want users to be able to specify additional options to the JVM, which will need to go before the -jar argument, as well as passing additional arguments to Jetty itself after the final start.jar argument

Other Java daemons like Elasticsearch have their own shell script that knows to look for $JAVA_OPTS or something similar, which is then generally wrapped with a Docker-specific entrypoint in the official images (as you're well aware).

[md5]

@tianon Also, as you may remember, this image originally called jetty.sh, which does know to look for Java options in an environment variable, but we moved away from it because it has too much init-specific logic in it.

[gregw]

@md5 Currently we have CMD and entry point:

ENTRYPOINT ["/docker-entrypoint.sh"]
CMD ["java","-Djava.io.tmpdir=/tmp/jetty","-jar","/usr/local/jetty/start.jar"]

Which has the benefit of making the default command line visible in the docker file.

However, let's say the distribution had a suitable jetty-start.sh script, then the docker file would end up being just:

ENTRYPOINT ["/docker-entrypoint.sh"]
CMD ["$JETTY_BASE/bin/jetty-start.sh"]

and we'd lose visibility of the default command line anyway.

So can we not just have an empty default CMD and move all the default command line into docker-entrypoint.sh:

ENTRYPOINT ["/docker-entrypoint.sh"]
CMD []

Users could then: set just JAVA_OPTIONS, which would be used by the default command line in docker-entrypoint.sh ; OR set their own CMD, which would be used directly by docker-entrypoint.sh as the command line if executable, or have java appended if not; OR set their own CMD and JAVA_OPTIONS, which would be used directly by docker-entrypoint.sh as the command line if executable, or have java $JAVA_OPTIONS appended if not.

[md5]

@gregw I think losing visibility of what's inside the jetty-start.sh script would be fine, assuming that script was the standard way to start Jetty and the ways to affect its behavior were well documented (as it is in the case of elasticsearch, neo4j, etc).

Since that's not currently the case and it doesn't seem to be on your roadmap, then I think doing something in the entrypoint makes sense. I'm not sure whether or not having a default CMD that mimics what happens in the entrypoint is a good idea or not. Since @tianon has written many of the official image entrypoints, I'd like to get his opinion as well.

[gregw]

@md5 as @tianon is currently silent, I've had a look at some of the docker images he works on:
haproxy has a Dockerfile that does include visible CMD:

ENTRYPOINT ["/docker-entrypoint.sh"]
CMD ["haproxy", "-f", "/usr/local/etc/haproxy/haproxy.cfg"]

But interesting the docker-entrypoint.sh script inspects and modifies that CMD. If it is left as haproxy then it will always replace the command with haproxy-systemd-wrapper and insert a -p argument. Importantly, it does not give the user the option of specifying their own command with a -p and avoiding the argument injection!

He's not committed to many other directly relevant projects, but looking further afield I can see other examples of entry points that modify the CMD from the docker file:

• elasticsearch
• cassandra
• redmine

So if we had an entry point that checked if $1 was java and if so it added $JAVA_OPTIONS, then we would not be an island.

I think I'll prepare a PR for this.

[md5]

Sounds good. It make take me a few days to review since we're coming into a holiday weekend here in the US

[gregw]

@md5 I created the PR. However I could not find a script generate-stackbrew-library.sh, but I did run the test/run.sh script

[md5]

It's in this repo: https://github.com/appropriate/docker-jetty/blob/master/generate-stackbrew-library.sh

[tianon]

Oh man, my inbox is such a train wreck; sorry I missed this entire conversation! Glad it ended up in a solid place in spite of me! 👍

[gregw]

@md5 Argh! I missed the alpine subdir! Plus my manually updated stackbrew listed a newer tag for the alpine version!

Standby....

[gregw]

@md5 Here is another PR fixing alpine and also removing the now unused ENV variables

[gregw]

Reopening as we need to update documentation.

[lfoppiano]

Hi, It's not that easy to follow the thread, at the end what would be the implemented solution?
Which version should I use?
Thanks
Luca

[md5]

@lfoppiano You would pass -Xmx to the container via the JAVA_OPTIONS environment variable:

$ docker run -d -p 80:8080 -p 443:8443 -e JAVA_OPTIONS='-Xmx1g' jetty

If you have a derived image that is FROM jetty, then you can use the ENV directive in your Dockerfile instead.

@gregw Would you still be willing to create a PR for the jetty image documentation so this information is easier to find?

[lfoppiano]

@md5 thanks.

To check that the parameters are correctly set, I need to connect to the container

docker exec -i -t 231b05ab1cbd /bin/bash

and check the environment variable:

root@231b05ab1cbd:/var/lib/jetty# echo $JAVA_OPTS
-Xmx4g

[md5]

There's not really a straightforward way to do that, but here are some options:

• You can call Runtime.getRuntime().maxMemory() in your code
• You can look at the java.lang:type=Memory MBean via JMX
• You can install a JDK inside your container and use jinfo via docker exec

The third option looks like it's a bit problematic however... I can't actually get jinfo to tell me anything, either as root or as the jetty user.

[md5]

I opened a PR for the docs here: docker-library/docs#709

[md5]

With the latest jre8 images, it should also be possible to use Docker's cgroup-based memory limits to configure the heap using the -XX:+UseCGroupMemoryLimitForHeap JVM option. See docker-library/openjdk#71 (comment)

Here's an example command that gives the container 600 megabytes and give the JVM half of that memory:

$ docker run -d -p 80:8080 -p 443:8443 --memory 600m \
  -e JAVA_OPTIONS='-XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -X:MaxRAMFraction=2'

I'm going to close this issue since the the way to set the heap size has been documented.