[SPARK-58678][CORE][TESTS] Add OpenHashMapBenchmark

[dongjoon-hyun]

What changes were proposed in this pull request?

This PR aims to add OpenHashMapBenchmark to measure the performance of
org.apache.spark.util.collection.OpenHashMap against java.util.HashMap in three scenarios
modeled on Spark's real usage:

• Insert: inserting 1M distinct String keys via update/put
• Aggregate: 5M changeValue/merge operations over 1M String keys (the typical
  aggregation pattern, e.g. Mode, countByValue)
• Lookup: looking up 1M String keys in random order

Why are the changes needed?

OpenHashMap claims that it is "about 5X faster than java.util.HashMap".

spark/core/src/main/scala/org/apache/spark/util/collection/OpenHashMap.scala

Line 24 in 8645e2c

* but not deletions. This map is about 5X faster than java.util.HashMap, while using much less

However, This claim dates from 2013 (pre-JDK 8), and there has been no benchmark in the repository to
verify it. On modern JDKs, java.util.HashMap has improved significantly. According to this benchmark, Java is much faster for Insert and Lookup.

• In case of Insert, java.util.HashMap is 4.6X, 5.5X, 6.5X faster in Java 17, 21, and 25, respectively.

• Note that OpenHashMap still uses about 2.2x less memory than java.util.HashMap for
  String -> Long entries thanks to its specialized primitive value storage. This benchmark
  provides a consistent way to track the trade-off across JDK versions and future improvements.

Does this PR introduce any user-facing change?

No. This is a test-only change.

How was this patch tested?

Manually ran the benchmark:

build/sbt "core/Test/runMain org.apache.spark.util.collection.OpenHashMapBenchmark"

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Fable 5

[dongjoon-hyun]

Could you review this PR, @uros-b ?

[uros-b]

Changes look good @dongjoon-hyun, but it seems that CI didn't run properly?

[dongjoon-hyun]

Thank you, @uros-b . CI is running here. So far, there is no issue because this is a new benchmark addition.

• https://github.com/dongjoon-hyun/spark/runs/93291137088

[dongjoon-hyun]

Let me merge this because the running CI is irrelevant to this PR and the benchmark generation itself is verified when generating the benchmark result already.

[dongjoon-hyun]

Merge Summary:

• merged into master 9a20f8b
• merged into branch-4.x 50a57cb
• merged into branch-4.3 62e79ed
• merged into branch-4.2 f56544a

Posted by merge_spark_pr.py