Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-36151][INFRA] MiMa updates after 3.2.0 release: bump previousSparkVersion and re-enable tests for Scala 2.13 artifacts #34306

Closed

Conversation

JoshRosen
Copy link
Contributor

What changes were proposed in this pull request?

This PR updates MiMa checks following Spark 3.2.0's release:

Why are the changes needed?

To ensure that MiMa checks cover new APIs added in Spark 3.2.0.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

$ dev/mima -Pscala-2.12
$ dev/mima -Pscala-2.13

@SparkQA
Copy link

SparkQA commented Oct 18, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48815/

@HyukjinKwon
Copy link
Member

Merged to master.

@SparkQA
Copy link

SparkQA commented Oct 18, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48815/

@SparkQA
Copy link

SparkQA commented Oct 18, 2021

Test build #144336 has finished for PR 34306 at commit b565feb.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@JoshRosen JoshRosen deleted the update-mima-after-3.2.0-release branch October 18, 2021 05:22
HyukjinKwon pushed a commit that referenced this pull request Oct 26, 2021
### What changes were proposed in this pull request?

This PR fixes the MiMa failure with Scala 2.13.

### Why are the changes needed?

SPARK-36151 (#34306) re-enabled MiMa for Scala 2.13 but it always fails in the scheduled build.
https://github.com/apache/spark/runs/3992588994?check_suite_focus=true#step:9:2303
The reason is that `dev/mima` and `MimaBuild` don't consider `2.13`.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

MiMa passed on GA with modified GA config (set `SCALA_PROFILE` to `scala2.13`).
https://github.com/sarutak/spark/runs/3998022279?check_suite_focus=true#step:9:19919

I also confirmed that `dev/change-scala-version.sh 2.12` and `dev/change-scala-version.sh 2.13` are idempotent.

Closes #34382 from sarutak/fix-mima-build-scala-2.13.

Authored-by: Kousuke Saruta <sarutak@oss.nttdata.com>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
3 participants