Skip to content

Benchmarks: make jmh.gradle run cleanly on JDK 17 (module opens, configurable heap, failOnError) #17330

Description

@vaquarkhan

Problem

The JMH harness fails on a clean JDK 17 checkout:

  1. Spark 4.x reflects into sun.util.calendar; without --add-opens the read-path benchmarks throw
    IllegalAccessException at warm-up (stack trace below).
  2. Heap is hard-coded to -Xmx32g; the full-scale PlanningBenchmark ...WithStats OOMs at that size.
  3. failOnError = true aborts the whole suite (and discards completed results) on one failure.

Reproduce (JDK 17, Corretto 17.0.19)

./gradlew -PsparkVersions=3.5 -PscalaVersion=2.12 \
  :iceberg-spark:iceberg-spark-extensions-3.5_2.12:jmh \
  -PjmhIncludeRegex=IcebergSourceParquetEqDeleteBenchmark \
  -PjmhOutputPath=build/reports/jmh/human.txt \
  -PjmhJsonOutputPath=build/reports/jmh/results.json

=> IllegalAccessException: sun.util.calendar.ZoneInfo

Example failure (from earlier EC2 run):

org.apache.spark.SparkException: Job aborted due to stage failure:
  Task 0 in stage 0.0 failed 1 times ...
java.lang.IllegalAccessException: symbolic reference class is not accessible:
  class sun.util.calendar.ZoneInfo, from interface
  org.apache.spark.sql.catalyst.util.SparkDateTimeUtils (unnamed module)
    at java.base/java.lang.invoke.MemberName.makeAccessException(MemberName.java:955)
    at org.apache.spark.sql.catalyst.util.SparkDateTimeUtils.toJavaDate(SparkDateTimeUtils.scala:304)
    at org.apache.iceberg.spark.source.IcebergSourceBenchmark.appendAsFile(IcebergSourceBenchmark.java:136)

Proposal

Add the JDK 17 --add-opens set to JMH jvmArgs, make heap configurable via -PjmhHeap (default 32g),
and make failOnError configurable via -PjmhFailOnError (default true). One file: jmh.gradle.

Evidence

  • Failure: read-path benchmarks crash at warm-up without sun.util.calendar open
  • Success: same suites complete on Corretto 17.0.19 / JMH 1.37 on EC2 r7i.4xlarge / r7i.8xlarge when the opens set and a larger heap (-PjmhHeap=220g for full-scale planning) are applied
  • Defaults preserved: heap stays 32g and failOnError stays true unless -P flags are passed

Sharing PR with fix

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions