Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[SPARK-34110][BUILD] Upgrade Zookeeper to 3.6.2
### What changes were proposed in this pull request? This PR upgrade Zookeeper to 3.6.2. ### Why are the changes needed? To make Spark running on jdk 14, otherwise: ``` 21/01/13 20:25:32,533 WARN [Driver-SendThread(apache-spark-zk-3.vip.hadoop.com:2181)] zookeeper.ClientCnxn:1164 : Session 0x0 for server apache-spark-zk-3.vip.hadoop.com/<unresolved>:2181, unexpected error, closing socket connection and attempting reconnect java.lang.IllegalArgumentException: Unable to canonicalize address apache-spark-zk-3.vip.hadoop.com/<unresolved>:2181 because it's not resolvable at org.apache.zookeeper.SaslServerPrincipal.getServerPrincipal(SaslServerPrincipal.java:65) at org.apache.zookeeper.SaslServerPrincipal.getServerPrincipal(SaslServerPrincipal.java:41) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1001) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060) ``` Please see [ZOOKEEPER-3779](https://issues.apache.org/jira/browse/ZOOKEEPER-3779) for more details. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Manual test: 1. Replace zookeeper-3.4.14.jar with zookeeper-3.6.2.jar and zookeeper-jute-3.6.2.jar 2. Run Spark on jdk 14. Hadoop 2.7 with HADOOP-12760, Hive 1.2.1 and Zookeeper server version is 3.4.6. Some key configurations: ``` # spark-defaults.conf spark.yarn.appMasterEnv.JAVA_HOME /apache/releases/jdk-14.0.2 spark.executorEnv.JAVA_HOME /apache/releases/jdk-14.0.2 # spark-env.sh export JAVA_HOME=/apache/releases/jdk-14.0.2 ``` Jenkins Tests - Hadoop 3.2: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134048/testReport - Hadoop 2.7: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134063/testReport Closes #31177 from wangyum/SPARK-34110. Authored-by: Yuming Wang <yumwang@ebay.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
- Loading branch information