[SPARK-40303][DOCS] Deprecate old Java 8 versions prior to 8u362

[wangyum]

What changes were proposed in this pull request?

This PR aims to deprecate old Java 8 versions prior to 8u362.

Why are the changes needed?

8u362 fixed a performance issue: openjdk/jdk8u-dev#161

Benchmark code:

val dir = "/tmp/spark/benchmark"
val N = 2000000
val columns = Range(0, 100).map(i => s"id % $i AS id$i")

spark.range(N).selectExpr(columns: _*).write.mode("Overwrite").parquet(dir)

val cnt = 60
val selectExps = columns.take(cnt).map(_.split(" ").last).map(c => s"count(distinct $c)")
val start = System.currentTimeMillis()
spark.read.parquet(dir).selectExpr(selectExps: _*).collect()
println(s"Benchmark result. Java version: ${System.getProperty("java.version")}. Time(ms): ${System.currentTimeMillis() - start}")

8u352 benchmark result:

export JAVA_HOME=/Users/yumwang/Downloads/zulu8.66.0.15-ca-jdk8.0.352-macosx_x64
export PATH=${JAVA_HOME}/bin:${PATH}

bin/spark-shell  --master "local[2]" -i benchmark.scala
Benchmark result. Java version: 1.8.0_352. Time(ms): 641155

8u362 benchmark result:

export JAVA_HOME=/Users/yumwang/Downloads/zulu8.68.0.19-ca-jdk8.0.362-macosx_x64
export PATH=${JAVA_HOME}/bin:${PATH}

bin/spark-shell  --master "local[2]" -i benchmark.scala
Benchmark result. Java version: 1.8.0_362. Time(ms): 79360

Does this PR introduce any user-facing change?

No.

How was this patch tested?

N/A.

[dongjoon-hyun]

Shall we simply deprecate, @wangyum ?

- Java 8 prior to version 8u201 support is deprecated as of Spark 3.2.0.
+ Java 8 prior to version 8u362 support is deprecated as of Spark 3.4.0.

[dongjoon-hyun]

Oh, does Zulu only have that released version, @wangyum ?

• https://bugs.openjdk.org/browse/JDK-8296506

I cannot find docker image and Adoptium (Temurin) Java yet.

• https://hub.docker.com/_/openjdk/tags?page=1&name=8u
• https://adoptium.net/temurin/archive/?version=8

[wangyum]

Oh, does Zulu only have that released version, @wangyum ?

• https://bugs.openjdk.org/browse/JDK-8296506

I cannot find docker image and Adoptium (Temurin) Java yet.

• https://hub.docker.com/_/openjdk/tags?page=1&name=8u
• https://adoptium.net/temurin/archive/?version=8

It is in progress:

[dongjoon-hyun]

+1, LGTM for Apache Spark 3.4.0.

cc @srowen , @HyukjinKwon , @xinrong-meng , @LuciferYang

[LuciferYang]

+1, LGTM, thanks @wangyum

[LuciferYang]

One problem is that GA is still using Temurin 8u352 for build and test. We need to wait for a while before running GA tasks using 8u362.

[LuciferYang]

Could you use 8u362 to run full UTs offline to check compatibility? Thanks ~ @wangyum

[dongjoon-hyun]

To @LuciferYang , I don't think this is a compatibility issue or any failure.

[LuciferYang]

@dongjoon-hyun Hmm...do you remember SPARK-40846? When we upgrade from 8u345 to 8u352 for GA testing, there are some time zone issue that need to be solved by changing the code, so I am not sure whether it is the right time to directly recommend a Java version without GA verification in the document. But maybe I'm too conservative.

[dongjoon-hyun]

Timezone issues are inevitable which we need to adjust the code in a regular basis, @LuciferYang .

[dongjoon-hyun]

BTW, we didn't cut the branch yet and we still have one month for Apache Spark 3.4.0 release. I'm considering that time period for this decision, @LuciferYang . You are also correct and being conservative is better if we don't have a room like that.

[LuciferYang]

Ok, plenty of time. I am fine to make this change

[dongjoon-hyun]

Now, both Zulu and Adoptiun(Temurin) are available. Thank you all. Merged to master.

[dongjoon-hyun]

FYI, GitHub Action is currently on the old version but jdk8u362 will be automatically applied in one or two weeks. There is no need for us to do something. I'll keep monitoring the version change.