-
Notifications
You must be signed in to change notification settings - Fork 28k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-37453][INFRA][SQL][TESTS] Split TPC-DS build in GitHub Actions #34698
Conversation
This comment has been minimized.
This comment has been minimized.
c37e174
to
4fd65df
Compare
This comment has been minimized.
This comment has been minimized.
2ffa2da
to
36b3473
Compare
joinConfSet.tail.foreach { conf => | ||
System.gc() // SPARK-37368 | ||
runQuery(queryString, goldenFile, conf.toSeq, true) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While I am here, I ended up with refactoring this code part here. No behavior change. Everything is as was.
This comment has been minimized.
This comment has been minimized.
cc @cloud-fan, @linhongliu-db and @jerqi |
36b3473
to
89b98d4
Compare
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Tests passed. Merged to master. |
Kubernetes integration test status failure |
Test build #145596 has finished for PR 34698 at commit
|
What changes were proposed in this pull request?
This is kind of a followup for #33510 and #34641. This PR proposes to split TPC-DS build in GitHub Actions.
Why are the changes needed?
Running these queries easily causes out-of-memory in GitHub Actions machines, and make the build flaky. We should deflake it.
Does this PR introduce any user-facing change?
No, dev-only.
How was this patch tested?
GitHub Actions in this PR should test it out.