Troubleshooting on Java program execution

[chongkong]

I'm suffering insufficient memory error when running Java code by isolate within docker container
[gist link]

Running code Main.java is a simple hello world program, and it works without error if executed outside of isolate.

I have no clue where the error had begun and how to deal with it. I wonder anybody's using isolate within docker container and runs non-native binary/codes (e.g. Java, Python, etc.)

---

Environment detail

• Macbook Pro 2016
• Docker 18.03.1-ce-mac65 (latest at the moment I'm writing)
  • Reserved CPU: 4
  • Reserved Memory: 4.0GiB
  • Swap: 1.0GiB
• latest isolate (commit 60caa49)
• Docker image openjdk:8u151-jre-alpine3.7

I also tried Java VM options such as

• stack size option -Xss16M
• heap size option -Xms16M, -Xmx16m
• code cache size -XX:ReservedCodeCacheSize=16M

[seirl]

Are you trying to limit the address space of the JVM? Don't do that, they assume they are able to use a lot of virtual memory when running. Limit the maximum resident memory instead.

[chongkong]

Well I also provided --mem option in isolate, does that limit the max RSS? Also why are limiting stack size and heap size through JVM options not recommended?

My primary goal is to make java program run (by any way / without limits) and I haven't succeeded it through isolate yet

[seirl]

--mem limits the address space, as documented here. You want to use --cg-mem to limit the resident memory. I advise you to never use --mem, it's most of the time useless to limit the address space of the programs you're running, except maybe for programs that don't do a lot of virtual memory mapping.

[chongkong]

Thanks a lot! Let me post a followup comment after i try it

[stefano-maggiolo]

For reference, here's how we use Java in CMS: https://github.com/cms-dev/cms/blob/master/cms/grading/languages/java_jdk.py

[chongkong]

I'm still getting the same exception even though I haven't made any resource limits and just ran java -version

$ isolate --init  # o.k.
$ isolate --run -- /usr/bin/java -version  # error

The error occurs immediately after I enter the command and error message is identical as the one in gist link above

#
# There is insufficient memory for the Java Runtime Environment to continue.
# Cannot create GC thread. Out of system resources.
# Can not save log file, dump to screen..
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Cannot create GC thread. Out of system resources.
# Possible reasons:
#   The system is out of physical RAM or swap space
#   In 32 bit mode, the process size limit was hit
# Possible solutions:
#   Reduce memory load on the system
#   Increase physical memory or swap space
#   Check if swap backing store is full
#   Use 64 bit Java on a 64 bit OS
#   Decrease Java heap size (-Xmx/-Xms)
#   Decrease number of Java threads
#   Decrease Java thread stack sizes (-Xss)
#   Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
#  Out of Memory Error (gcTaskThread.cpp:48), pid=2, tid=0x00002b451f523b88
#
# JRE version:  (8.0_151-b12) (build )
# Java VM: OpenJDK 64-Bit Server VM (25.151-b12 mixed mode linux-amd64 compressed oops)
# Derivative: IcedTea 3.6.0
# Distribution: Custom build (Tue Nov 21 11:22:36 GMT 2017)
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#

Some other notes...

• Docker image is based on openjdk/8u151-jdk-alpine3.7, running in privileged mode
• It works well if I just run /usr/bin/java -version without isolate.
• Specifying following JVM options yielded same results
  • GC Thread options -XX:ParallelGCThreads=2
  • Reserved code cache size option -XX:ReservedCodeCacheSize=16M
  • Heap and stack size option Xss??M, Xmx??M
• ulimit -a inside the container (not isolate sandbox) gives you the following result

-f: file size (blocks)             unlimited
-t: cpu time (seconds)             unlimited
-d: data seg size (kb)             unlimited
-s: stack size (kb)                8192
-c: core file size (blocks)        0
-m: resident set size (kb)         unlimited
-l: locked memory (kb)             82000
-p: processes                      unlimited
-n: file descriptors               1048576
-v: address space (kb)             unlimited
-w: locks                          unlimited
-e: scheduling priority            0
-r: real-time priority             0

[chongkong]

Well, it worked after I set --processes=X where X >= 11
I'm trying to figure out why so much default processes are required (even if I specify -XX:ParallelGCThreads=1) but finally it worked

[seirl]

I was going to say one of the pitfalls of isolate is running it without options thinking it won't add any restrictions, because the --processes option is the only one that have a limit by default.

I'd strongly advise to never use --processes to isolate the JVM, as the errors it will produce will be very very hard to see. At some point, we had Java just hang without dumping any state to stderr, and we couldn't debug the issue at all until I just randomly bumped the process limit from 20 to 100 and it worked.

There are not only GC threads, also JIT compilation threads, you can also specify ConcGCThreads etc... Java is just really bad at being isolated (https://jaxenter.com/nobody-puts-java-container-139373.html) either because of the number of processes required, or because of its habit of looking at the available physical RAM to see how much virtual memory it is going to allocate (it's really a percentage of the total RAM...).

For what it's worth, it's supposed to be better in Java 10: https://blog.docker.com/2018/04/improved-docker-container-integration-with-java-10/

[gollux]

I do not think it is wise not to limit the number of processes at all. You really do not want the sandboxed program to run 1000 threads at once. Better set --processes to a high, but still reasonable number, like 100. Java memory management is awful, but fixing the amount of memory by -X<something> options usually helps.