Skip to content

[SPARK-33179][TESTS] Switch default Hadoop profile in run-tests.py#30090

Closed
williamhyun wants to merge 1 commit intoapache:masterfrom
williamhyun:SPARK-33179
Closed

[SPARK-33179][TESTS] Switch default Hadoop profile in run-tests.py#30090
williamhyun wants to merge 1 commit intoapache:masterfrom
williamhyun:SPARK-33179

Conversation

@williamhyun
Copy link
Copy Markdown
Member

@williamhyun williamhyun commented Oct 19, 2020

What changes were proposed in this pull request?

This PR aims to switch the default Hadoop profile from hadoop2.7 to hadoop3.2 in dev/run-tests.py when it's running in local or GitHub Action environments.

Why are the changes needed?

The default Hadoop version is 3.2. We had better be consistent.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Manually.

BEFORE

% dev/run-tests.py
Cannot install SparkR as R was not found in PATH
[info] Using build tool sbt with Hadoop profile hadoop2.7 and Hive profile hive2.3 under environment local

AFTER

% dev/run-tests.py
Cannot install SparkR as R was not found in PATH
[info] Using build tool sbt with Hadoop profile hadoop3.2 and Hive profile hive2.3 under environment local

@williamhyun williamhyun changed the title [SPARK-33179][TESTS] Switch default Hadoop version in run-tests.py [SPARK-33179][TESTS] Switch default Hadoop profile in run-tests.py Oct 19, 2020
@SparkQA
Copy link
Copy Markdown

SparkQA commented Oct 19, 2020

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34589/

@SparkQA
Copy link
Copy Markdown

SparkQA commented Oct 19, 2020

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34589/

@HyukjinKwon
Copy link
Copy Markdown
Member

Merged to master.

@SparkQA
Copy link
Copy Markdown

SparkQA commented Oct 19, 2020

Test build #129982 has finished for PR 30090 at commit d7c5630.

  • This patch fails due to an unknown error code, -9.
  • This patch merges cleanly.
  • This patch adds no public classes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants