Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-35227][BUILD] Update the resolver for spark-packages in SparkSubmit #32346

Closed
wants to merge 1 commit into from
Closed

[SPARK-35227][BUILD] Update the resolver for spark-packages in SparkSubmit #32346

wants to merge 1 commit into from

Conversation

bozhang2820
Copy link
Contributor

What changes were proposed in this pull request?

This change is to use repos.spark-packages.org instead of Bintray as the repository service for spark-packages.

Why are the changes needed?

The change is needed because Bintray will no longer be available from May 1st.

Does this PR introduce any user-facing change?

This should be transparent for users who use SparkSubmit.

How was this patch tested?

Tested running spark-shell with --packages manually.

@github-actions github-actions bot added the CORE label Apr 26, 2021
HyukjinKwon
HyukjinKwon approved these changes Apr 26, 2021
View reviewed changes
Copy link
Member

@HyukjinKwon HyukjinKwon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Shall we file a JIRA?

@HyukjinKwon
Copy link
Member

@viirya, are you preparing RC? This can be backported to Spark 2.4 too .. although this isn't really a blocker. If you're not preparing, I will just backport. If you're I will just don't port it back to branch-2.4.

@bozhang2820 bozhang2820 changed the title Update the resolver for spark-packages in SparkSubmit [SPARK-35227] Update the resolver for spark-packages in SparkSubmit Apr 26, 2021
@HyukjinKwon HyukjinKwon changed the title [SPARK-35227] Update the resolver for spark-packages in SparkSubmit [SPARK-35227][BUILD] Update the resolver for spark-packages in SparkSubmit Apr 26, 2021
@bozhang2820
Copy link
Contributor Author

Looks good. Shall we file a JIRA?

Yes. Filed SPARK-35227 for this.

@HyukjinKwon
Copy link
Member

cc @srowen too fyi

@viirya
Copy link
Member

viirya commented Apr 26, 2021

@viirya, are you preparing RC? This can be backported to Spark 2.4 too .. although this isn't really a blocker. If you're not preparing, I will just backport. If you're I will just don't port it back to branch-2.4.

Not yet as I have some network issue locally. You can just backport it. :)

@SparkQA
Copy link

SparkQA commented Apr 26, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42459/

@SparkQA
Copy link

SparkQA commented Apr 26, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42459/

@SparkQA
Copy link

SparkQA commented Apr 26, 2021

Test build #137937 has finished for PR 32346 at commit 4a678ff.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@bozhang2820
Copy link
Contributor Author

Should we do a maintenance release on the currently supported version for this?

@srowen
Copy link
Member

srowen commented Apr 27, 2021

This will be backported to 2.4; not sure if we'd cut releases for this specially rather than their usual cadences, but, 2.4.8 is coming shortly, and 3.0/3.1 will have some maintenance release in months I'd imagine.

HyukjinKwon pushed a commit that referenced this pull request Apr 27, 2021
@bozhang2820 @HyukjinKwon
[SPARK-35227][BUILD] Update the resolver for spark-packages in SparkS…
cb1d802 
…ubmit

### What changes were proposed in this pull request?
This change is to use repos.spark-packages.org instead of Bintray as the repository service for spark-packages.

### Why are the changes needed?
The change is needed because Bintray will no longer be available from May 1st.

### Does this PR introduce _any_ user-facing change?
This should be transparent for users who use SparkSubmit.

### How was this patch tested?
Tested running spark-shell with --packages manually.

Closes #32346 from bozhang2820/replace-bintray.

Authored-by: Bo Zhang <bo.zhang@databricks.com>
Signed-off-by: hyukjinkwon <gurwls223@apache.org>
(cherry picked from commit f738fe0)
Signed-off-by: hyukjinkwon <gurwls223@apache.org>
HyukjinKwon pushed a commit that referenced this pull request Apr 27, 2021
@bozhang2820 @HyukjinKwon
[SPARK-35227][BUILD] Update the resolver for spark-packages in SparkS…
5ca67d6 
…ubmit

This change is to use repos.spark-packages.org instead of Bintray as the repository service for spark-packages.

The change is needed because Bintray will no longer be available from May 1st.

This should be transparent for users who use SparkSubmit.

Tested running spark-shell with --packages manually.

Closes #32346 from bozhang2820/replace-bintray.

Authored-by: Bo Zhang <bo.zhang@databricks.com>
Signed-off-by: hyukjinkwon <gurwls223@apache.org>
(cherry picked from commit f738fe0)
Signed-off-by: hyukjinkwon <gurwls223@apache.org>
@HyukjinKwon
Copy link
Member

Merged to master, branch-3.1, branch-3.0 and branch-2.4.

HyukjinKwon pushed a commit that referenced this pull request Apr 27, 2021
@bozhang2820 @HyukjinKwon
[SPARK-35227][BUILD] Update the resolver for spark-packages in SparkS…
183eb75 
…ubmit

This change is to use repos.spark-packages.org instead of Bintray as the repository service for spark-packages.

The change is needed because Bintray will no longer be available from May 1st.

This should be transparent for users who use SparkSubmit.

Tested running spark-shell with --packages manually.

Closes #32346 from bozhang2820/replace-bintray.

Authored-by: Bo Zhang <bo.zhang@databricks.com>
Signed-off-by: hyukjinkwon <gurwls223@apache.org>
(cherry picked from commit f738fe0)
Signed-off-by: hyukjinkwon <gurwls223@apache.org>
@bozhang2820 bozhang2820 deleted the replace-bintray branch May 11, 2021 06:11
flyrain pushed a commit to flyrain/spark that referenced this pull request Sep 21, 2021
@bozhang2820 @viirya
[SPARK-35227][BUILD] Update the resolver for spark-packages in SparkS…
97d5475 
…ubmit

### What changes were proposed in this pull request?
This change is to use repos.spark-packages.org instead of Bintray as the repository service for spark-packages.

### Why are the changes needed?
The change is needed because Bintray will no longer be available from May 1st.

### Does this PR introduce _any_ user-facing change?
This should be transparent for users who use SparkSubmit.

### How was this patch tested?
Tested running spark-shell with --packages manually.

Closes apache#32346 from bozhang2820/replace-bintray.

Authored-by: Bo Zhang <bo.zhang@databricks.com>
Signed-off-by: hyukjinkwon <gurwls223@apache.org>
(cherry picked from commit f738fe0)
Signed-off-by: hyukjinkwon <gurwls223@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@viirya viirya viirya approved these changes

@srowen srowen srowen approved these changes

@HyukjinKwon HyukjinKwon HyukjinKwon approved these changes

@dongjoon-hyun dongjoon-hyun dongjoon-hyun approved these changes

Assignees
No one assigned
Labels
CORE
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@bozhang2820 @HyukjinKwon @viirya @SparkQA @srowen @dongjoon-hyun