Skip to content

[SPARK-36451][BUILD] Ivy skips looking for source and doc pom#33678

Closed
cxzl25 wants to merge 2 commits intoapache:masterfrom
cxzl25:SPARK-36451
Closed

[SPARK-36451][BUILD] Ivy skips looking for source and doc pom#33678
cxzl25 wants to merge 2 commits intoapache:masterfrom
cxzl25:SPARK-36451

Conversation

@cxzl25
Copy link
Contributor

@cxzl25 cxzl25 commented Aug 8, 2021

What changes were proposed in this pull request?

Because SPARK-35863 Upgrade Ivy to 2.5.0, it supports skip searching the source and doc pom, but the remote repo will still be queried at present.

Why are the changes needed?

Can improve the speed of some UT, such as IsolatedClientLoader#downloadVersion, no need to find source and doc pom

Does this PR introduce any user-facing change?

No

How was this patch tested?

manual UT

@github-actions github-actions bot added the CORE label Aug 8, 2021
@srowen
Copy link
Member

srowen commented Aug 17, 2021

Are there any other implications to not looking for source or doc POMs? I can't quite think of them, just checking if there are any possible breakages.

@cxzl25
Copy link
Contributor Author

cxzl25 commented Aug 17, 2021

We can improve the execution speed of UT, because ivy will now connect to the remote repo, analyze three pom, source, doc pom, and only download the binary jar.

@srowen
Copy link
Member

srowen commented Aug 17, 2021

I get that; that wasn't the question though. Are there any other implications of not downloading these, that are bad?

@cxzl25
Copy link
Contributor Author

cxzl25 commented Aug 17, 2021

AFAIK, there is no bad effect. Maybe you can trigger a test to verify, especially the UT execution speed of multiple versions of hive client.

@srowen
Copy link
Member

srowen commented Aug 17, 2021

Jenkins test this please

@cxzl25
Copy link
Contributor Author

cxzl25 commented Aug 17, 2021

org.apache.ivy.plugins.parser.m2.PomModuleDescriptorParser # addSourcesAndJavadocArtifactsIfPresent

https://github.com/apache/ant-ivy/blob/8e2bdfb302f1e3e6e20fc727e1853c5edabdf7d4/src/java/org/apache/ivy/plugins/parser/m2/PomModuleDescriptorParser.java#L363-L377

@SparkQA
Copy link

SparkQA commented Aug 17, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47065/

@SparkQA
Copy link

SparkQA commented Aug 17, 2021

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47065/

@SparkQA
Copy link

SparkQA commented Aug 17, 2021

Test build #142563 has finished for PR 33678 at commit 0ff77e6.

  • This patch fails due to an unknown error code, -9.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Aug 17, 2021

Test build #142562 has finished for PR 33678 at commit 0ff77e6.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@srowen srowen closed this in a1ecf83 Aug 18, 2021
@srowen
Copy link
Member

srowen commented Aug 18, 2021

Merged to master

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants