Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-35178][BUILD] Use new Apache 'closer.lua' syntax to obtain Maven #32277

Closed
wants to merge 2 commits into from

Conversation

srowen
Copy link
Member

@srowen srowen commented Apr 21, 2021

What changes were proposed in this pull request?

Use new Apache 'closer.lua' syntax to obtain Maven

Why are the changes needed?

The current closer.lua redirector, which redirects to download Maven from a local mirror, has a new syntax. build/mvn does not work properly otherwise now.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Manual testing.

@github-actions github-actions bot added the BUILD label Apr 21, 2021
@srowen
Copy link
Member Author

srowen commented Apr 21, 2021

Note we should backport this all the way to 2.4, as it would affect all builds

@srowen
Copy link
Member Author

srowen commented Apr 21, 2021

Wait a beat - https://issues.apache.org/jira/browse/INFRA-21767 says there is a simpler updated URL to use.

@srowen srowen changed the title [SPARK-35178][BUILD] Use Apache 'closer' service to obtain mirror host [SPARK-35178][BUILD] Use new Apache 'closer.lua' syntax to obtain Maven Apr 21, 2021
@SparkQA
Copy link

SparkQA commented Apr 21, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42272/

@SparkQA
Copy link

SparkQA commented Apr 21, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42272/

@SparkQA
Copy link

SparkQA commented Apr 21, 2021

Kubernetes integration test unable to build dist.

exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42275/

@SparkQA
Copy link

SparkQA commented Apr 21, 2021

Test build #137745 has finished for PR 32277 at commit 5169b16.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Apr 21, 2021

Test build #137748 has finished for PR 32277 at commit 9a79232.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM. Thank you, @srowen .
I checked that this works correctly in your GitHub Action job (linter).
And, also checked manually.

$ build/mvn clean
exec: curl --silent --show-error -L https://downloads.lightbend.com/scala/2.12.10/scala-2.12.10.tgz
exec: curl --silent --show-error -L https://www.apache.org/dyn/closer.lua/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz?action=download
Using `mvn` from path: /Users/dongjoon/PRS/SPARK-PR-32277/build/apache-maven-3.6.3/bin/mvn
...

dongjoon-hyun pushed a commit that referenced this pull request Apr 22, 2021
### What changes were proposed in this pull request?

Use new Apache 'closer.lua' syntax to obtain Maven

### Why are the changes needed?

The current closer.lua redirector, which redirects to download Maven from a local mirror, has a new syntax. build/mvn does not work properly otherwise now.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Manual testing.

Closes #32277 from srowen/SPARK-35178.

Authored-by: Sean Owen <srowen@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
(cherry picked from commit 6860efe)
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
dongjoon-hyun pushed a commit that referenced this pull request Apr 22, 2021
### What changes were proposed in this pull request?

Use new Apache 'closer.lua' syntax to obtain Maven

### Why are the changes needed?

The current closer.lua redirector, which redirects to download Maven from a local mirror, has a new syntax. build/mvn does not work properly otherwise now.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Manual testing.

Closes #32277 from srowen/SPARK-35178.

Authored-by: Sean Owen <srowen@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
(cherry picked from commit 6860efe)
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
@dongjoon-hyun
Copy link
Member

Merged to master/3.1/3.0. For 2.4, could you make a backport PR?

@dongjoon-hyun
Copy link
Member

Oh, 3.1/3.0 has Zinc installation. I'll make a follow-up for 3.1/3.0.

@dongjoon-hyun
Copy link
Member

Here is the follow-up to update Zinc.

@srowen srowen deleted the SPARK-35178 branch June 1, 2021 13:48
flyrain pushed a commit to flyrain/spark that referenced this pull request Sep 21, 2021
### What changes were proposed in this pull request?

Use new Apache 'closer.lua' syntax to obtain Maven

### Why are the changes needed?

The current closer.lua redirector, which redirects to download Maven from a local mirror, has a new syntax. build/mvn does not work properly otherwise now.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Manual testing.

Closes apache#32277 from srowen/SPARK-35178.

Authored-by: Sean Owen <srowen@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
(cherry picked from commit 6860efe)
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
3 participants