[SPARK-34111][BUILD] Deconflict the jars jakarta.servlet-api-4.0.3.jar and javax.servlet-api-3.1.0.jar by yaooqinn · Pull Request #31185 · apache/spark

yaooqinn · 2021-01-15T04:00:04Z

What changes were proposed in this pull request?

Use the right name for the artifact javax.servlet-api in hadoop 3.2 profile

Why are the changes needed?

fix deps issue

Does this PR introduce any user-facing change?

no

How was this patch tested?

should pass existing jenkins
I have manually tested a spark application with the javax.servlet-api-3.1.0.jar deleted and it worked fine

…r and javax.servlet-api-3.1.0.jar

yaooqinn · 2021-01-15T04:04:41Z

cc @HyukjinKwon @dongjoon-hyun thanks for reviewing

dongjoon-hyun · 2021-01-15T04:06:05Z

dev/deps/spark-deps-hadoop-3.2-hive-2.3

 javassist/3.25.0-GA//javassist-3.25.0-GA.jar
 javax.inject/1//javax.inject-1.jar
 javax.jdo/3.2.0-m3//javax.jdo-3.2.0-m3.jar
-javax.servlet-api/3.1.0//javax.servlet-api-3.1.0.jar


dongjoon-hyun · 2021-01-15T04:07:18Z

Thank you, @yaooqinn . Yes, let's see the test result.

HyukjinKwon · 2021-01-15T04:09:03Z

Nice, thank you @yaooqinn!

HyukjinKwon

The fix itself looks good. should be good to go if tests pass

SparkQA · 2021-01-15T05:28:29Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38662/

yaooqinn · 2021-01-15T05:51:15Z

retest this please

SparkQA · 2021-01-15T05:54:35Z

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38662/

yaooqinn · 2021-01-15T06:43:40Z

- decommission workers ensure that shuffle output is regenerated even with shuffle service *** FAILED *** (20 seconds, 783 milliseconds)
[info]   5 did not equal 4 Expected 4 tasks but got List(0:0:0:0-SUCCESS, 0:0:1:0-FAILED, 0:0:1:1-SUCCESS, 0:1:0:0-SUCCESS, 1:0:0:0-SUCCESS) (DecommissionWorkerSuite.scala:190)
[info]   org.scalatest.exceptions.TestFailedException:
[info]   at org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:472)
[info]   at org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:471)
[info]   at org.scalatest.Assertions$.newAssertionFailedException(Assertions.scala:1231)
[info]   at org.scalatest.Assertions$AssertionsHelper.macroAssert(Assertions.scala:1295)
[info]   at org.apache.spark.deploy.DecommissionWorkerSuite.$anonfun$new$6(DecommissionWorkerSuite.scala:190)

seems just a flaky test itself and not related to this change

HyukjinKwon · 2021-01-15T06:50:43Z

retest this please

SparkQA · 2021-01-15T06:59:32Z

Test build #134090 has started for PR 31185 at commit f8f12a0.

SparkQA · 2021-01-15T07:15:45Z

Test build #134076 has finished for PR 31185 at commit f8f12a0.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2021-01-15T07:41:21Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38676/

SparkQA · 2021-01-15T07:43:26Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38673/

SparkQA · 2021-01-15T08:15:34Z

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38673/

SparkQA · 2021-01-15T08:18:43Z

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38676/

HyukjinKwon · 2021-01-15T08:58:00Z

retest this please

SparkQA · 2021-01-15T10:33:39Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38682/

SparkQA · 2021-01-15T10:58:56Z

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38682/

HyukjinKwon · 2021-01-15T11:18:37Z

retest this please

SparkQA · 2021-01-15T12:33:30Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38687/

SparkQA · 2021-01-15T13:09:16Z

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38687/

SparkQA · 2021-01-15T13:54:31Z

Test build #134084 has finished for PR 31185 at commit f8f12a0.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

HyukjinKwon · 2021-01-15T13:58:36Z

retest this please

yaooqinn · 2021-01-15T14:20:15Z

look ok in both mvn and sbt tests

HyukjinKwon · 2021-01-15T14:34:25Z

Hm, wait. Let's wait for GA results. Seems like this particular job (which is related):

Build modules: streaming, sql-kafka-0-10, streaming-kafka-0-10, mllib-local, mllib, yarn, mesos, kubernetes, hadoop-cloud, spark-ganglia-lgpl (JDK 8, hadoop3.2, hive2.3)

keeps pending.

yaooqinn · 2021-01-15T14:59:09Z

retest this please

SparkQA · 2021-01-15T15:33:29Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38693/

SparkQA · 2021-01-15T16:10:12Z

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38693/

SparkQA · 2021-01-15T16:39:31Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38697/

SparkQA · 2021-01-15T17:14:50Z

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38697/

SparkQA · 2021-01-15T17:50:46Z

Test build #134096 has finished for PR 31185 at commit f8f12a0.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2021-01-15T18:19:59Z

Test build #134112 has finished for PR 31185 at commit f8f12a0.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2021-01-15T18:56:11Z

Test build #134114 has finished for PR 31185 at commit f8f12a0.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2021-01-15T20:14:30Z

Test build #134101 has finished for PR 31185 at commit f8f12a0.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

dongjoon-hyun · 2021-01-15T22:28:44Z

~~Sorry, but could you rebase once more, @yaooqinn ?~~

It seems that this is superseded by SPARK-33212. Could you check that?

SparkQA · 2021-01-15T23:15:52Z

Test build #134109 has finished for PR 31185 at commit f8f12a0.

This patch fails from timeout after a configured wait of 500m.
This patch merges cleanly.
This patch adds no public classes.

yaooqinn · 2021-01-16T03:45:37Z

yes, I checked and saw the servlet-api gone away, we don't need this PR anymore @dongjoon-hyun.

dongjoon-hyun · 2021-01-16T21:58:52Z

Thank you for the confirmation, @yaooqinn .

dongjoon-hyun · 2021-01-16T22:01:26Z

Instead, could you open a PR to branch-3.1, @yaooqinn ?

dongjoon-hyun · 2021-01-16T22:02:23Z

SPARK-33212 is only applied at master branch for Apache Spark 3.2.0. So, SPARK-34111 is still a blocker for Apache Spark 3.1.0 release.

yaooqinn · 2021-01-17T02:12:42Z

OK

[SPARK-34111][BUILD] Deconflict the jars jakarta.servlet-api-4.0.3.ja…

f8f12a0

…r and javax.servlet-api-3.1.0.jar

dongjoon-hyun reviewed Jan 15, 2021

View reviewed changes

HyukjinKwon reviewed Jan 15, 2021

View reviewed changes

github-actions bot added the BUILD label Jan 15, 2021

yaooqinn changed the title ~~[SPARK-34111][BUILD] Deconflict the jars jakarta.servlet-api-4.0.3.jar and javax.servlet-api-3.1.0.jar~~ [SPARK-34111][BUILD][test-maven] Deconflict the jars jakarta.servlet-api-4.0.3.jar and javax.servlet-api-3.1.0.jar Jan 15, 2021

HyukjinKwon approved these changes Jan 15, 2021

View reviewed changes

yaooqinn changed the title ~~[SPARK-34111][BUILD][test-maven] Deconflict the jars jakarta.servlet-api-4.0.3.jar and javax.servlet-api-3.1.0.jar~~ [SPARK-34111][BUILD] Deconflict the jars jakarta.servlet-api-4.0.3.jar and javax.servlet-api-3.1.0.jar Jan 15, 2021

srowen approved these changes Jan 15, 2021

View reviewed changes

dongjoon-hyun closed this Jan 16, 2021

Conversation

yaooqinn commented Jan 15, 2021

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Uh oh!

yaooqinn commented Jan 15, 2021

Uh oh!

dongjoon-hyun Jan 15, 2021

Choose a reason for hiding this comment

Uh oh!

dongjoon-hyun commented Jan 15, 2021

Uh oh!

HyukjinKwon commented Jan 15, 2021

Uh oh!

HyukjinKwon left a comment

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Jan 15, 2021

Uh oh!

yaooqinn commented Jan 15, 2021

Uh oh!

SparkQA commented Jan 15, 2021

Uh oh!

yaooqinn commented Jan 15, 2021

Uh oh!

HyukjinKwon commented Jan 15, 2021

Uh oh!

SparkQA commented Jan 15, 2021

Uh oh!

SparkQA commented Jan 15, 2021

Uh oh!

SparkQA commented Jan 15, 2021

Uh oh!

SparkQA commented Jan 15, 2021

Uh oh!

SparkQA commented Jan 15, 2021

Uh oh!

SparkQA commented Jan 15, 2021

Uh oh!

HyukjinKwon commented Jan 15, 2021

Uh oh!

SparkQA commented Jan 15, 2021

Uh oh!

SparkQA commented Jan 15, 2021

Uh oh!

HyukjinKwon commented Jan 15, 2021

Uh oh!

SparkQA commented Jan 15, 2021

Uh oh!

SparkQA commented Jan 15, 2021

Uh oh!

SparkQA commented Jan 15, 2021

Uh oh!

HyukjinKwon commented Jan 15, 2021

Uh oh!

yaooqinn commented Jan 15, 2021

Uh oh!

HyukjinKwon commented Jan 15, 2021

Uh oh!

yaooqinn commented Jan 15, 2021

Uh oh!

SparkQA commented Jan 15, 2021

Uh oh!

SparkQA commented Jan 15, 2021

Uh oh!

SparkQA commented Jan 15, 2021

Uh oh!

SparkQA commented Jan 15, 2021

Uh oh!

SparkQA commented Jan 15, 2021

Uh oh!

SparkQA commented Jan 15, 2021

Uh oh!

SparkQA commented Jan 15, 2021

Uh oh!

SparkQA commented Jan 15, 2021

Uh oh!

dongjoon-hyun commented Jan 15, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

dongjoon-hyun commented Jan 15, 2021 •

edited

Loading