fix(compose): drop JDK 8 GC options that break arm64 images - #127
fix(compose): drop JDK 8 GC options that break arm64 images#127barbatos2011 wants to merge 1 commit into
Conversation
c269433 to
440ab18
Compare
|
|
||
| Flags in `command` are used for java-tron start-up arguments: | ||
| - `-jvm` used for java virtual machine, the parameters must be enclosed in double quotes and braces. `"{-Xmx14g -Xms12g}"` sets the maximum and initial heap size to 14GB and 12GB respectively. If you want to set up a long run FullNode, please use the best practice jvm flags with `"{-Xmx14g -Xmn2g XX:ReservedCodeCacheSize=256m -XX:MetaspaceSize=256m -XX:MaxMetaspaceSize=512m -XX:MaxDirectMemorySize=1G -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:gc.log -XX:+UseConcMarkSweepGC -XX:NewRatio=2 -XX:+CMSScavengeBeforeRemark -XX:+ParallelRefProcEnabled -XX:+HeapDumpOnOutOfMemoryError -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70}"`. | ||
| - `-jvm` used for java virtual machine, the parameters must be enclosed in double quotes and braces. `"{-Xmx14g -Xms12g}"` sets the maximum and initial heap size to 14GB and 12GB respectively. If you want to set up a long run FullNode, please use the best practice jvm flags with `"{-Xmx14g -Xmn2g -XX:ReservedCodeCacheSize=256m -XX:MetaspaceSize=256m -XX:MaxMetaspaceSize=512m -XX:MaxDirectMemorySize=1G -XX:+ParallelRefProcEnabled -XX:+HeapDumpOnOutOfMemoryError}"`. Do not add garbage collector or GC logging options here. The image ships an architecture-specific `bin/java-tron.vmoptions` that already selects a collector: CMS on the amd64/JDK 8 image, ZGC on the arm64/JDK 17 image. JDK 8 only flags such as `-XX:+UseConcMarkSweepGC` and `-Xloggc:` are rejected by JDK 17 and stop the arm64 container from starting. |
There was a problem hiding this comment.
[SHOULD] -Xloggc is still accepted by JDK 17 as a deprecated alias that maps to unified logging. The failure here is caused by the target directory being missing or unwritable, rather than by -Xloggc itself being unsupported. See the Oracle JDK 17 java documentation.
There was a problem hiding this comment.
Correct, thanks — that wording conflated two different failure modes and I have fixed it in a347ea2.
Verified on the arm64 image (tronprotocol/java-tron:latest, JDK 17.0.19):
$ docker run --rm --entrypoint java <img> -Xloggc:/tmp/gc.log -version
[0.000s][warning][gc] -Xloggc is deprecated. Will use -Xlog:gc:/tmp/gc.log instead.
openjdk version "17.0.19" 2026-04-21 # starts normally
$ docker run --rm --entrypoint java <img> -Xloggc:./nosuchdir/gc.log -version
[0.000s][warning][gc] -Xloggc is deprecated. Will use -Xlog:gc:./nosuchdir/gc.log instead.
[0.000s][error ][logging] Error opening log file './nosuchdir/gc.log': No such file or directory
Error: Could not create the Java Virtual Machine.
$ docker run --rm --entrypoint java <img> -XX:+UseConcMarkSweepGC -version
Unrecognized VM option 'UseConcMarkSweepGC'
Error: Could not create the Java Virtual Machine.So the two flags fail for unrelated reasons: -XX:+UseConcMarkSweepGC was removed in JDK 14 and is rejected outright, while -Xloggc: still resolves to unified logging and only aborts because the target directory is missing. That distinction matters here because /java-tron/logs does not exist in the image at JVM startup — java-tron creates it later, at application startup, which is after JVM logging has already initialised. On JDK 8 the same situation is a warning and the node keeps running.
The README now states both reasons separately. The commit message and PR description already described the directory-dependent behaviour correctly; only that one sentence was wrong.
| services: | ||
| tron-node-mainnet: | ||
| container_name: tron-node | ||
| image: tronprotocol/java-tron:GreatVoyage-v4.8.0 # Add specific tag if needed |
There was a problem hiding this comment.
[SHOULD] Change to latest for better compatibility with future changes?
There was a problem hiding this comment.
Happy to, but I would rather leave this to you, because the tag choice here is a deployment policy call rather than a compatibility one, and two things suggest it is not blocking this PR.
First, this pin cannot reach the failure this PR fixes. tronprotocol/java-tron:GreatVoyage-v4.8.0 is a single-image manifest, not a manifest list:
$ docker buildx imagetools inspect tronprotocol/java-tron:GreatVoyage-v4.8.0
MediaType: application/vnd.docker.distribution.manifest.v2+json # no platform listSo it can never resolve to linux/arm64, and the JDK 17 crash does not apply to this file. What does apply after this change is only the GC log relocation, from the host-mounted logs/ to /java-tron/gc.log inside the container.
Second, this is the production CodeDeploy path (appspec.yml → start_service.sh), and validate_service.sh only polls the API for five minutes, so it would not notice a bad image swap. Moving a pinned production artifact to latest makes the deployment track whatever was published most recently, which is usually the argument for pinning rather than against it.
If you would still like it changed I will do it in this PR, or as a separate one so the deployment policy change is reviewable on its own. Your call.
| services: | ||
| tron-node-nile: | ||
| container_name: tron-node | ||
| image: java-tron:Nile_4.8.0 # Add specific tag if needed |
There was a problem hiding this comment.
[SHOULD] Change to latest for better compatibility with future changes?
There was a problem hiding this comment.
This one has an extra wrinkle: java-tron:Nile_4.8.0 has no organisation prefix, so it resolves to docker.io/library/java-tron:Nile_4.8.0, which is not published:
$ docker manifest inspect java-tron:Nile_4.8.0
errors:
denied: requested access to the resource is deniedIt can only be an image built locally on the deployment host under that name. Changing it to latest would point at library/java-tron:latest, a different repository altogether, so that would not be a compatibility improvement — it would most likely stop resolving.
If the intent is to track the published Nile image, the change would be to tronnile/java-tron:latest, which is a genuine repository change rather than a tag bump, and worth doing deliberately.
Also worth noting that the pipeline scripts here (appspec.yml, start_service.sh, prepare_environment.sh) all reference docker-compose.fullnode.main.yml only, so this nile file does not appear to be exercised by CodeDeploy at all.
Same as the mainnet file: happy to change it if you want, but I would rather not fold a repository change into this PR.
| - ./datadir:/java-tron/data # mount a local directory to make the blocks data persistent. | ||
| - ./logs/tron-witness1:/java-tron/logs # map to host logs | ||
| command: > | ||
| -jvm "{-Xmx12g -Xmn2g -XX:+UseConcMarkSweepGC -Xloggc:./logs/gc.log}" -c /java-tron/conf/private_net_config_witness1.conf -d /java-tron/data -w |
There was a problem hiding this comment.
[NIT] Until a change similar to [tronprotocol/java-tron#6918](tronprotocol/java-tron#6918) takes effect, extra care is needed because gc.log is no longer mounted to the host directory. As a result, the GC logs will be lost if the container is removed or recreated.
There was a problem hiding this comment.
Agreed, and thanks for linking it — that is exactly the gap tronprotocol/java-tron#6918 closes. It points both vmoptions files at ./logs/gc.log and creates the directory in the launcher before the JVM starts, so once it lands and images are rebuilt, gc.log lands in the mounted logs/ directory next to tron.log on both architectures, with no further change needed here.
The mkdir has to live in the launcher rather than in a RUN mkdir in the Dockerfiles, because JVM logging initialises before java-tron creates ./logs; handling it in the launcher also covers consumers of the plain distribution zip. Verified by applying only the vmoptions half to the arm64 image:
[0.000s][error][logging] Error opening log file './logs/gc.log': No such file or directory
Error: Could not create the Java Virtual Machine.Until then, the GC log is retrievable with docker cp <container>:/java-tron/gc.log . while the container exists.
I kept the two changes in separate PRs so this one stays reviewable on its own and does not block on a java-tron release. If you would prefer the interim gap closed here instead, say so and I will look at it.
The published tronprotocol/java-tron image is a multi-arch manifest whose
arm64 variant runs JDK 17, while the amd64 variant runs JDK 8. Every compose
file hard-coded JDK 8 era GC options inside the -jvm argument:
-XX:+UseConcMarkSweepGC removed in JDK 14, so the JVM refuses to start
-Xloggc:<path> mapped to unified logging in JDK 9+, which fails
hard when the target directory does not yet exist
On an arm64 host the quick start documented in README.md therefore exits
within one second with exit code 1:
Unrecognized VM option 'UseConcMarkSweepGC'
Error: Could not create the Java Virtual Machine.
Rather than adding a parallel set of arm64 compose files, let the image
decide. bin/java-tron.vmoptions is generated per architecture at java-tron
build time and already selects CMS on amd64 and ZGC on arm64, so the compose
files only need to size the heap. One file now works on both architectures.
single_node/README.md carried the same options in two places, including the
string offered as "best practice jvm flags" for a long running FullNode, so
an operator following the documentation reproduced the crash the compose
change removes. Both are updated, with a note that GC selection belongs to
the image. That paragraph also had a malformed entry, XX:ReservedCodeCacheSize
without its leading dash, which java parses as a main class name and rejects
with ClassNotFoundException; it is corrected here.
GC logging is unchanged in substance but moves location: it is now written to
/java-tron/gc.log inside the container as configured by the image, instead of
the previous ./logs/gc.log on the host.
Verified on linux/arm64 against tronprotocol/java-tron:latest (JDK 17.0.19):
the node starts, serves /wallet/getnodeinfo and reports a ZHeap memory pool.
The remaining option set of all ten compose files, and the new README flag
string, were checked against both the arm64 image and a local JDK 8.
440ab18 to
a347ea2
Compare
Problem
tronprotocol/java-tron:latestis a multi-arch manifest. Its arm64 variant runs JDK 17, while the amd64 variant runs JDK 8 (seetools/docker/Dockerfile.arm64, which installsopenjdk-17-jre-headless).Every compose file in this repository hard-codes JDK 8 era GC options in the
-jvmargument, and there is no arm64 variant of any of them. Two of those options are fatal on JDK 17:-XX:+UseConcMarkSweepGCUnrecognized VM option— the JVM refuses to start.-Xloggc:<path>As a result, on any ARM host (Apple Silicon, AWS Graviton, ARM servers) the quick start documented in
README.mdfails immediately:The same image started with no
-jvmargument runs normally, so the image is fine — the compose arguments are what stop it.docker-compose-quick-start.ymladditionally hits the-Xloggcfailure, because it mounts no volumes and/java-tron/logsdoes not exist in the image at JVM startup (java-tron creates it later, at application startup, which is too late for the JVM logger).Fix
Instead of adding a parallel set of arm64 compose files, let the image decide.
bin/java-tron.vmoptionsis generated per architecture at java-tron build time and already selects the right collector — CMS on amd64, ZGC on arm64. The compose files only need to size the heap. Removing the GC and GC-logging options makes one file correct on both architectures, and removes a class of "outer arguments fight the image's own configuration" bugs rather than just this instance.13 occurrences across 10 compose files, keeping all memory options untouched:
-XX:+PrintGCis dropped as well; the image'svmoptionsalready configures more detailed GC logging on both architectures.Documentation
single_node/README.mdcarried the same options in two places, so an operator following the documentation reproduced the crash the compose change removes:-XX:+UseConcMarkSweepGC.Unrecognized VM option 'PrintGCDateStamps'.Both now list only architecture-neutral memory and diagnostic options, with a note that GC selection belongs to the image. That paragraph also had a malformed entry,
XX:ReservedCodeCacheSize=256mmissing its leading dash, whichjavaparses as a main class name:Error: Could not find or load main class XX:ReservedCodeCacheSize=256mIt is corrected here.
Behaviour change
GC logging continues, but moves from the host's
./logs/gc.logto/java-tron/gc.loginside the container, since it is now configured by the image. Retrieve it withdocker cp tron-node:/java-tron/gc.log .This is addressed by tronprotocol/java-tron#6918, which points both
vmoptionsfiles at./logs/gc.logand creates the directory in the launcher before the JVM starts. Once that lands and images are rebuilt,gc.logappears in the mountedlogs/directory next totron.logon both architectures, with no further change needed here.That companion change also has to create the directory itself rather than relying on a
mkdirin the Dockerfiles here: JVM logging initialises before java-tron creates./logs, and on JDK 9+ a missing directory aborts startup. Handling it in the launcher covers every consumer of the distribution, including the plain zip, so no Dockerfile change is proposed in this PR.Verification
Host
linux/arm64, imagetronprotocol/java-tron:latest(openjdk 17.0.19,vmoptions= ZGC).End-to-end, through the real entrypoint:
The
ZHeappool is the point: the collector is chosen by the image according to its own JDK, so the same compose file yields CMS on amd64 and ZGC on arm64.The remaining option set of all ten compose files was additionally validated against the image's JDK 17 combined with the image's own
vmoptions; all are accepted. The pre-change option set was checked as a negative control and fails as described above. The new README flag string was validated against both the arm64 image (JDK 17 + ZGC vmoptions) and a local JDK 8 with the amd64 vmoptions.amd64 was not re-run. The change is low risk there: the image's
vmoptionsalready contains the same CMS options, so collector behaviour is unchanged and only the GC log path moves.Suggested follow-up
CI currently runs goss tests only on the build architecture, so it cannot catch this class of problem. An arm64 smoke test (start container, wait for HTTP 200, stop) would have.