Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-14818] Post-2.0 MiMa exclusion and build changes #15061

Closed
wants to merge 1 commit into from

Conversation

JoshRosen
Copy link
Contributor

This patch makes a handful of post-Spark-2.0 MiMa exclusion and build updates. It should be merged to master and a subset of it should be picked into branch-2.0 in order to test Spark 2.0.1-SNAPSHOT.

// [SPARK-14743] Improve delegation token handling in secure cluster
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.deploy.SparkHadoopUtil.getTimeFromNowToRenewal")
)
// SPARK-17096: Improve exception string reported through the StreamingQueryListener
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/cc @tdas, @zsxwing, I've added exclusions for the this change from #14675. I wanted to ping you here in case this change was unintentional and you think we'll need to restore compatibility.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JoshRosen That's the intentional broken. Thanks!

@JoshRosen JoshRosen changed the title [SPARK-14818] Post-2.0 MiMa exclucsion and build changes [SPARK-14818] Post-2.0 MiMa exclusion and build changes Sep 12, 2016
@yhuai
Copy link
Contributor

yhuai commented Sep 12, 2016

LGTM

@SparkQA
Copy link

SparkQA commented Sep 12, 2016

Test build #65268 has finished for PR 15061 at commit 1224e75.

  • This patch fails from timeout after a configured wait of 250m.
  • This patch merges cleanly.
  • This patch adds no public classes.

@JoshRosen
Copy link
Contributor Author

I think that the test timeout is unrelated (since it occurred in PySpark tests and those are unaffected by changes to MiMa excludes), so I'm going to merge this now and will cherry-pick to branch-2.0.

@asfgit asfgit closed this in 7c51b99 Sep 12, 2016
asfgit pushed a commit that referenced this pull request Sep 12, 2016
This patch makes a handful of post-Spark-2.0 MiMa exclusion and build updates. It should be merged to master and a subset of it should be picked into branch-2.0 in order to test Spark 2.0.1-SNAPSHOT.

- Remove the ` sketch`, `mllibLocal`, and `streamingKafka010` from the list of excluded subprojects so that MiMa checks them.
- Remove now-unnecessary special-case handling of the Kafka 0.8 artifact in `mimaSettings`.
- Move the exclusion added in SPARK-14743 from `v20excludes` to `v21excludes`, since that patch was only merged into master and not branch-2.0.
- Add exclusions for an API change introduced by SPARK-17096 / #14675.
- Add missing exclusions for the `o.a.spark.internal` and `o.a.spark.sql.internal` packages.

Author: Josh Rosen <joshrosen@databricks.com>

Closes #15061 from JoshRosen/post-2.0-mima-changes.

(cherry picked from commit 7c51b99)
Signed-off-by: Josh Rosen <joshrosen@databricks.com>
@JoshRosen JoshRosen deleted the post-2.0-mima-changes branch September 12, 2016 22:35
wgtmac pushed a commit to wgtmac/spark that referenced this pull request Sep 19, 2016
This patch makes a handful of post-Spark-2.0 MiMa exclusion and build updates. It should be merged to master and a subset of it should be picked into branch-2.0 in order to test Spark 2.0.1-SNAPSHOT.

- Remove the ` sketch`, `mllibLocal`, and `streamingKafka010` from the list of excluded subprojects so that MiMa checks them.
- Remove now-unnecessary special-case handling of the Kafka 0.8 artifact in `mimaSettings`.
- Move the exclusion added in SPARK-14743 from `v20excludes` to `v21excludes`, since that patch was only merged into master and not branch-2.0.
- Add exclusions for an API change introduced by SPARK-17096 / apache#14675.
- Add missing exclusions for the `o.a.spark.internal` and `o.a.spark.sql.internal` packages.

Author: Josh Rosen <joshrosen@databricks.com>

Closes apache#15061 from JoshRosen/post-2.0-mima-changes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants