Skip to content

fix(compose): drop JDK 8 GC options that break arm64 images - #127

Open
barbatos2011 wants to merge 1 commit into
tronprotocol:developfrom
barbatos2011:fix/jvm-gc-args-arm64
Open

fix(compose): drop JDK 8 GC options that break arm64 images#127
barbatos2011 wants to merge 1 commit into
tronprotocol:developfrom
barbatos2011:fix/jvm-gc-args-arm64

Conversation

@barbatos2011

@barbatos2011 barbatos2011 commented Aug 15, 2026

Copy link
Copy Markdown

Problem

tronprotocol/java-tron:latest is a multi-arch manifest. Its arm64 variant runs JDK 17, while the amd64 variant runs JDK 8 (see tools/docker/Dockerfile.arm64, which installs openjdk-17-jre-headless).

Every compose file in this repository hard-codes JDK 8 era GC options in the -jvm argument, and there is no arm64 variant of any of them. Two of those options are fatal on JDK 17:

Option JDK 17 behaviour
-XX:+UseConcMarkSweepGC Removed in JDK 14. Unrecognized VM option — the JVM refuses to start.
-Xloggc:<path> Mapped to unified logging. Unlike JDK 8, initialization fails hard when the target directory does not exist.

As a result, on any ARM host (Apple Silicon, AWS Graviton, ARM servers) the quick start documented in README.md fails immediately:

$ docker compose -f single_node/docker-compose-quick-start.yml up -d
$ docker ps -a --filter name=tron-node --format '{{.Status}}'
Exited (1)

$ docker logs tron-node
Unrecognized VM option 'UseConcMarkSweepGC'
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

The same image started with no -jvm argument runs normally, so the image is fine — the compose arguments are what stop it.

docker-compose-quick-start.yml additionally hits the -Xloggc failure, because it mounts no volumes and /java-tron/logs does not exist in the image at JVM startup (java-tron creates it later, at application startup, which is too late for the JVM logger).

Fix

Instead of adding a parallel set of arm64 compose files, let the image decide.

bin/java-tron.vmoptions is generated per architecture at java-tron build time and already selects the right collector — CMS on amd64, ZGC on arm64. The compose files only need to size the heap. Removing the GC and GC-logging options makes one file correct on both architectures, and removes a class of "outer arguments fight the image's own configuration" bugs rather than just this instance.

13 occurrences across 10 compose files, keeping all memory options untouched:

-      -jvm "{-Xmx14g -Xmn2g -XX:MaxDirectMemorySize=1G -XX:+UseConcMarkSweepGC -XX:+PrintGC -Xloggc:./logs/gc.log}"
+      -jvm "{-Xmx14g -Xmn2g -XX:MaxDirectMemorySize=1G}"

-XX:+PrintGC is dropped as well; the image's vmoptions already configures more detailed GC logging on both architectures.

Documentation

single_node/README.md carried the same options in two places, so an operator following the documentation reproduced the crash the compose change removes:

  • line 91 — a compose example passing -XX:+UseConcMarkSweepGC.
  • line 101 — the string offered as "best practice jvm flags" for a long running FullNode, carrying the full JDK 8 CMS set. Verified against the arm64 image, that string fails at Unrecognized VM option 'PrintGCDateStamps'.

Both now list only architecture-neutral memory and diagnostic options, with a note that GC selection belongs to the image. That paragraph also had a malformed entry, XX:ReservedCodeCacheSize=256m missing its leading dash, which java parses as a main class name:

Error: Could not find or load main class XX:ReservedCodeCacheSize=256m

It is corrected here.

Behaviour change

GC logging continues, but moves from the host's ./logs/gc.log to /java-tron/gc.log inside the container, since it is now configured by the image. Retrieve it with docker cp tron-node:/java-tron/gc.log .

This is addressed by tronprotocol/java-tron#6918, which points both vmoptions files at ./logs/gc.log and creates the directory in the launcher before the JVM starts. Once that lands and images are rebuilt, gc.log appears in the mounted logs/ directory next to tron.log on both architectures, with no further change needed here.

That companion change also has to create the directory itself rather than relying on a mkdir in the Dockerfiles here: JVM logging initialises before java-tron creates ./logs, and on JDK 9+ a missing directory aborts startup. Handling it in the launcher covers every consumer of the distribution, including the plain zip, so no Dockerfile change is proposed in this PR.

Verification

Host linux/arm64, image tronprotocol/java-tron:latest (openjdk 17.0.19, vmoptions = ZGC).

End-to-end, through the real entrypoint:

status        : Up
codeVersion   : 4.8.2.1
javaVersion   : 17.0.19
jvmFreeMemory : 14164164608     <- -Xmx14g in effect
memory pool   : ZHeap           <- ZGC selected by the image

The ZHeap pool is the point: the collector is chosen by the image according to its own JDK, so the same compose file yields CMS on amd64 and ZGC on arm64.

The remaining option set of all ten compose files was additionally validated against the image's JDK 17 combined with the image's own vmoptions; all are accepted. The pre-change option set was checked as a negative control and fails as described above. The new README flag string was validated against both the arm64 image (JDK 17 + ZGC vmoptions) and a local JDK 8 with the amd64 vmoptions.

amd64 was not re-run. The change is low risk there: the image's vmoptions already contains the same CMS options, so collector behaviour is unchanged and only the GC log path moves.

Suggested follow-up

CI currently runs goss tests only on the build architecture, so it cannot catch this class of problem. An arm64 smoke test (start container, wait for HTTP 200, stop) would have.

Comment thread single_node/README.md Outdated

Flags in `command` are used for java-tron start-up arguments:
- `-jvm` used for java virtual machine, the parameters must be enclosed in double quotes and braces. `"{-Xmx14g -Xms12g}"` sets the maximum and initial heap size to 14GB and 12GB respectively. If you want to set up a long run FullNode, please use the best practice jvm flags with `"{-Xmx14g -Xmn2g XX:ReservedCodeCacheSize=256m -XX:MetaspaceSize=256m -XX:MaxMetaspaceSize=512m -XX:MaxDirectMemorySize=1G -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:gc.log -XX:+UseConcMarkSweepGC -XX:NewRatio=2 -XX:+CMSScavengeBeforeRemark -XX:+ParallelRefProcEnabled -XX:+HeapDumpOnOutOfMemoryError -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70}"`.
- `-jvm` used for java virtual machine, the parameters must be enclosed in double quotes and braces. `"{-Xmx14g -Xms12g}"` sets the maximum and initial heap size to 14GB and 12GB respectively. If you want to set up a long run FullNode, please use the best practice jvm flags with `"{-Xmx14g -Xmn2g -XX:ReservedCodeCacheSize=256m -XX:MetaspaceSize=256m -XX:MaxMetaspaceSize=512m -XX:MaxDirectMemorySize=1G -XX:+ParallelRefProcEnabled -XX:+HeapDumpOnOutOfMemoryError}"`. Do not add garbage collector or GC logging options here. The image ships an architecture-specific `bin/java-tron.vmoptions` that already selects a collector: CMS on the amd64/JDK 8 image, ZGC on the arm64/JDK 17 image. JDK 8 only flags such as `-XX:+UseConcMarkSweepGC` and `-Xloggc:` are rejected by JDK 17 and stop the arm64 container from starting.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SHOULD] -Xloggc is still accepted by JDK 17 as a deprecated alias that maps to unified logging. The failure here is caused by the target directory being missing or unwritable, rather than by -Xloggc itself being unsupported. See the Oracle JDK 17 java documentation.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, thanks — that wording conflated two different failure modes and I have fixed it in a347ea2.

Verified on the arm64 image (tronprotocol/java-tron:latest, JDK 17.0.19):

$ docker run --rm --entrypoint java <img> -Xloggc:/tmp/gc.log -version
[0.000s][warning][gc] -Xloggc is deprecated. Will use -Xlog:gc:/tmp/gc.log instead.
openjdk version "17.0.19" 2026-04-21        # starts normally

$ docker run --rm --entrypoint java <img> -Xloggc:./nosuchdir/gc.log -version
[0.000s][warning][gc] -Xloggc is deprecated. Will use -Xlog:gc:./nosuchdir/gc.log instead.
[0.000s][error  ][logging] Error opening log file './nosuchdir/gc.log': No such file or directory
Error: Could not create the Java Virtual Machine.

$ docker run --rm --entrypoint java <img> -XX:+UseConcMarkSweepGC -version
Unrecognized VM option 'UseConcMarkSweepGC'
Error: Could not create the Java Virtual Machine.

So the two flags fail for unrelated reasons: -XX:+UseConcMarkSweepGC was removed in JDK 14 and is rejected outright, while -Xloggc: still resolves to unified logging and only aborts because the target directory is missing. That distinction matters here because /java-tron/logs does not exist in the image at JVM startup — java-tron creates it later, at application startup, which is after JVM logging has already initialised. On JDK 8 the same situation is a warning and the node keeps running.

The README now states both reasons separately. The commit message and PR description already described the directory-dependent behaviour correctly; only that one sentence was wrong.

services:
tron-node-mainnet:
container_name: tron-node
image: tronprotocol/java-tron:GreatVoyage-v4.8.0 # Add specific tag if needed

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SHOULD] Change to latest for better compatibility with future changes?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to, but I would rather leave this to you, because the tag choice here is a deployment policy call rather than a compatibility one, and two things suggest it is not blocking this PR.

First, this pin cannot reach the failure this PR fixes. tronprotocol/java-tron:GreatVoyage-v4.8.0 is a single-image manifest, not a manifest list:

$ docker buildx imagetools inspect tronprotocol/java-tron:GreatVoyage-v4.8.0
MediaType: application/vnd.docker.distribution.manifest.v2+json     # no platform list

So it can never resolve to linux/arm64, and the JDK 17 crash does not apply to this file. What does apply after this change is only the GC log relocation, from the host-mounted logs/ to /java-tron/gc.log inside the container.

Second, this is the production CodeDeploy path (appspec.ymlstart_service.sh), and validate_service.sh only polls the API for five minutes, so it would not notice a bad image swap. Moving a pinned production artifact to latest makes the deployment track whatever was published most recently, which is usually the argument for pinning rather than against it.

If you would still like it changed I will do it in this PR, or as a separate one so the deployment policy change is reviewable on its own. Your call.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, leave it as is now.

services:
tron-node-nile:
container_name: tron-node
image: java-tron:Nile_4.8.0 # Add specific tag if needed

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SHOULD] Change to latest for better compatibility with future changes?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one has an extra wrinkle: java-tron:Nile_4.8.0 has no organisation prefix, so it resolves to docker.io/library/java-tron:Nile_4.8.0, which is not published:

$ docker manifest inspect java-tron:Nile_4.8.0
errors:
denied: requested access to the resource is denied

It can only be an image built locally on the deployment host under that name. Changing it to latest would point at library/java-tron:latest, a different repository altogether, so that would not be a compatibility improvement — it would most likely stop resolving.

If the intent is to track the published Nile image, the change would be to tronnile/java-tron:latest, which is a genuine repository change rather than a tag bump, and worth doing deliberately.

Also worth noting that the pipeline scripts here (appspec.yml, start_service.sh, prepare_environment.sh) all reference docker-compose.fullnode.main.yml only, so this nile file does not appear to be exercised by CodeDeploy at all.

Same as the mainnet file: happy to change it if you want, but I would rather not fold a repository change into this PR.

- ./datadir:/java-tron/data # mount a local directory to make the blocks data persistent.
- ./logs/tron-witness1:/java-tron/logs # map to host logs
command: >
-jvm "{-Xmx12g -Xmn2g -XX:+UseConcMarkSweepGC -Xloggc:./logs/gc.log}" -c /java-tron/conf/private_net_config_witness1.conf -d /java-tron/data -w

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[NIT] Until a change similar to [tronprotocol/java-tron#6918](tronprotocol/java-tron#6918) takes effect, extra care is needed because gc.log is no longer mounted to the host directory. As a result, the GC logs will be lost if the container is removed or recreated.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, and thanks for linking it — that is exactly the gap tronprotocol/java-tron#6918 closes. It points both vmoptions files at ./logs/gc.log and creates the directory in the launcher before the JVM starts, so once it lands and images are rebuilt, gc.log lands in the mounted logs/ directory next to tron.log on both architectures, with no further change needed here.

The mkdir has to live in the launcher rather than in a RUN mkdir in the Dockerfiles, because JVM logging initialises before java-tron creates ./logs; handling it in the launcher also covers consumers of the plain distribution zip. Verified by applying only the vmoptions half to the arm64 image:

[0.000s][error][logging] Error opening log file './logs/gc.log': No such file or directory
Error: Could not create the Java Virtual Machine.

Until then, the GC log is retrievable with docker cp <container>:/java-tron/gc.log . while the container exists.

I kept the two changes in separate PRs so this one stays reviewable on its own and does not block on a java-tron release. If you would prefer the interim gap closed here instead, say so and I will look at it.

The published tronprotocol/java-tron image is a multi-arch manifest whose
arm64 variant runs JDK 17, while the amd64 variant runs JDK 8. Every compose
file hard-coded JDK 8 era GC options inside the -jvm argument:

  -XX:+UseConcMarkSweepGC  removed in JDK 14, so the JVM refuses to start
  -Xloggc:<path>           mapped to unified logging in JDK 9+, which fails
                           hard when the target directory does not yet exist

On an arm64 host the quick start documented in README.md therefore exits
within one second with exit code 1:

  Unrecognized VM option 'UseConcMarkSweepGC'
  Error: Could not create the Java Virtual Machine.

Rather than adding a parallel set of arm64 compose files, let the image
decide. bin/java-tron.vmoptions is generated per architecture at java-tron
build time and already selects CMS on amd64 and ZGC on arm64, so the compose
files only need to size the heap. One file now works on both architectures.

single_node/README.md carried the same options in two places, including the
string offered as "best practice jvm flags" for a long running FullNode, so
an operator following the documentation reproduced the crash the compose
change removes. Both are updated, with a note that GC selection belongs to
the image. That paragraph also had a malformed entry, XX:ReservedCodeCacheSize
without its leading dash, which java parses as a main class name and rejects
with ClassNotFoundException; it is corrected here.

GC logging is unchanged in substance but moves location: it is now written to
/java-tron/gc.log inside the container as configured by the image, instead of
the previous ./logs/gc.log on the host.

Verified on linux/arm64 against tronprotocol/java-tron:latest (JDK 17.0.19):
the node starts, serves /wallet/getnodeinfo and reports a ZHeap memory pool.
The remaining option set of all ten compose files, and the new README flag
string, were checked against both the arm64 image and a local JDK 8.
@barbatos2011
barbatos2011 force-pushed the fix/jvm-gc-args-arm64 branch from 440ab18 to a347ea2 Compare August 15, 2026 14:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants