Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-33439][INFRA] Use SERIAL_SBT_TESTS=1 for SQL modules #30365

Closed
wants to merge 3 commits into from
Closed

[SPARK-33439][INFRA] Use SERIAL_SBT_TESTS=1 for SQL modules #30365

wants to merge 3 commits into from

Conversation

dongjoon-hyun
Copy link
Member

@dongjoon-hyun dongjoon-hyun commented Nov 13, 2020

What changes were proposed in this pull request?

This PR aims to decrease the parallelism of SQL module like Hive module.

Why are the changes needed?

GitHub Action sql - slow tests become flaky.

Does this PR introduce any user-facing change?

No. This is dev-only feature.
Although this will increase the running time, but it's better than flakiness.

How was this patch tested?

Pass the GitHub Action stably.

@github-actions github-actions bot added the INFRA label Nov 13, 2020
@dongjoon-hyun
Copy link
Member Author

Hi, @HyukjinKwon . Could you review this PR?
This PR aims to stabilize the recent flakiness of sql - slow tests module.

@dongjoon-hyun
Copy link
Member Author

Thank you, @HyukjinKwon . I triggered three runs and I hope it will show stableness a little.

@HyukjinKwon
Copy link
Member

@dongjoon-hyun, sure. I think it's fine to merge and see if it fixes as well. I will leave it to you.

@dongjoon-hyun
Copy link
Member Author

Got it. Thank you always!

@SparkQA
Copy link

SparkQA commented Nov 13, 2020

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35640/

@SparkQA
Copy link

SparkQA commented Nov 13, 2020

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35640/

@dongjoon-hyun
Copy link
Member Author

dongjoon-hyun commented Nov 13, 2020

sql - slow tests succeeded in all three run.
Merged to master/3.0

dongjoon-hyun added a commit that referenced this pull request Nov 13, 2020
### What changes were proposed in this pull request?

This PR aims to decrease the parallelism of `SQL` module like `Hive` module.

### Why are the changes needed?

GitHub Action `sql - slow tests` become flaky.
- https://github.com/apache/spark/runs/1393670291
- https://github.com/apache/spark/runs/1393088031

### Does this PR introduce _any_ user-facing change?

No. This is dev-only feature.
Although this will increase the running time, but it's better than flakiness.

### How was this patch tested?

Pass the GitHub Action stably.

Closes #30365 from dongjoon-hyun/SPARK-33439.

Authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit a70a2b0)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
@dongjoon-hyun dongjoon-hyun deleted the SPARK-33439 branch November 13, 2020 05:20
@SparkQA
Copy link

SparkQA commented Nov 13, 2020

Test build #131034 has finished for PR 30365 at commit 1db4c78.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants