Problem
The JMH harness fails on a clean JDK 17 checkout:
- Spark 4.x reflects into
sun.util.calendar; without --add-opens the read-path benchmarks throw
IllegalAccessException at warm-up (stack trace below).
- Heap is hard-coded to
-Xmx32g; the full-scale PlanningBenchmark ...WithStats OOMs at that size.
failOnError = true aborts the whole suite (and discards completed results) on one failure.
Reproduce (JDK 17, Corretto 17.0.19)
./gradlew -PsparkVersions=3.5 -PscalaVersion=2.12 \
:iceberg-spark:iceberg-spark-extensions-3.5_2.12:jmh \
-PjmhIncludeRegex=IcebergSourceParquetEqDeleteBenchmark \
-PjmhOutputPath=build/reports/jmh/human.txt \
-PjmhJsonOutputPath=build/reports/jmh/results.json
=> IllegalAccessException: sun.util.calendar.ZoneInfo
Example failure (from earlier EC2 run):
org.apache.spark.SparkException: Job aborted due to stage failure:
Task 0 in stage 0.0 failed 1 times ...
java.lang.IllegalAccessException: symbolic reference class is not accessible:
class sun.util.calendar.ZoneInfo, from interface
org.apache.spark.sql.catalyst.util.SparkDateTimeUtils (unnamed module)
at java.base/java.lang.invoke.MemberName.makeAccessException(MemberName.java:955)
at org.apache.spark.sql.catalyst.util.SparkDateTimeUtils.toJavaDate(SparkDateTimeUtils.scala:304)
at org.apache.iceberg.spark.source.IcebergSourceBenchmark.appendAsFile(IcebergSourceBenchmark.java:136)
Proposal
Add the JDK 17 --add-opens set to JMH jvmArgs, make heap configurable via -PjmhHeap (default 32g),
and make failOnError configurable via -PjmhFailOnError (default true). One file: jmh.gradle.
Evidence
- Failure: read-path benchmarks crash at warm-up without
sun.util.calendar open
- Success: same suites complete on Corretto 17.0.19 / JMH 1.37 on EC2
r7i.4xlarge / r7i.8xlarge when the opens set and a larger heap (-PjmhHeap=220g for full-scale planning) are applied
- Defaults preserved: heap stays
32g and failOnError stays true unless -P flags are passed
Sharing PR with fix
Problem
The JMH harness fails on a clean JDK 17 checkout:
sun.util.calendar; without--add-opensthe read-path benchmarks throwIllegalAccessExceptionat warm-up (stack trace below).-Xmx32g; the full-scalePlanningBenchmark...WithStatsOOMs at that size.failOnError = trueaborts the whole suite (and discards completed results) on one failure.Reproduce (JDK 17, Corretto 17.0.19)
=>
IllegalAccessException: sun.util.calendar.ZoneInfoExample failure (from earlier EC2 run):
Proposal
Add the JDK 17
--add-opensset to JMHjvmArgs, make heap configurable via-PjmhHeap(default32g),and make
failOnErrorconfigurable via-PjmhFailOnError(defaulttrue). One file:jmh.gradle.Evidence
sun.util.calendaropenr7i.4xlarge/r7i.8xlargewhen the opens set and a larger heap (-PjmhHeap=220gfor full-scale planning) are applied32gandfailOnErrorstaystrueunless-Pflags are passedSharing PR with fix