[SPARK-32695][INFRA] Explicitly cache and hash 'build' directly in GitHub Actions #29536
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This comment has been minimized.
This comment has been minimized.
This reverts commit cfe5e92.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This reverts commit 1f4706e.
This comment has been minimized.
This comment has been minimized.
HyukjinKwon
changed the title
|
cc @dongjoon-hyun and @gengliangwang can you take a look when you're available? |
|
Test build #127872 has finished for PR 29536 at commit
|
| @@ -99,10 +99,14 @@ jobs: | |||
| run: git merge --progress --ff-only origin/${{ github.event.inputs.target }} | |||
| # Cache local repositories. Note that GitHub Actions cache has a 2G limit. | |||
| - name: Cache Scala, SBT, Maven and Zinc | |||
| uses: actions/cache@v1 | |||
| uses: actions/cache@v2 | |||
There was a problem hiding this comment.
why is the difference between v1 and v2?
There was a problem hiding this comment.
In order to allow multiple file path (https://github.com/actions/cache#whats-new). It is also to make it consistent in this file. cache@v2 is used in all places at this file except here.
There was a problem hiding this comment.
Previously, other files like build/mvn and build/sbt are also cached and overwritten. So, when you have some changes there, they are ignored.
maropu
reviewed
Aug 25, 2020
| build/zinc-* | ||
| build/scala-* | ||
| build/*.jar | ||
| key: build-${{ hashFiles('**/pom.xml', 'project/build.properties', 'build/mvn', 'build/sbt', 'build/sbt-launch-lib.bash', 'build/spark-build-info') }} |
There was a problem hiding this comment.
(Ah, I see. glob patterns like build/*' cannot be used in hashFilesnow...)
|
Looked around the GH actions docs and this update looks resonable to me. |
|
Thanks @maropu and @gengliangwang! |
|
Merged to master. branch-3.0 and branch-2.4. |
HyukjinKwon
added a commit
that referenced
this pull request
Aug 26, 2020
…tHub Actions ### What changes were proposed in this pull request? This PR proposes to explicitly cache and hash the files/directories under 'build' for SBT and Zinc at GitHub Actions. Otherwise, it can end up with overwriting `build` directory. See also #29286 (comment) Previously, other files like `build/mvn` and `build/sbt` are also cached and overwritten. So, when you have some changes there, they are ignored. ### Why are the changes needed? To make GitHub Actions build stable. ### Does this PR introduce _any_ user-facing change? No, dev-only. ### How was this patch tested? The builds in this PR test it out. Closes #29536 from HyukjinKwon/SPARK-32695. Authored-by: HyukjinKwon <gurwls223@apache.org> Signed-off-by: HyukjinKwon <gurwls223@apache.org> (cherry picked from commit b07e742) Signed-off-by: HyukjinKwon <gurwls223@apache.org>
HyukjinKwon
added a commit
that referenced
this pull request
Aug 26, 2020
…tHub Actions ### What changes were proposed in this pull request? This PR proposes to explicitly cache and hash the files/directories under 'build' for SBT and Zinc at GitHub Actions. Otherwise, it can end up with overwriting `build` directory. See also #29286 (comment) Previously, other files like `build/mvn` and `build/sbt` are also cached and overwritten. So, when you have some changes there, they are ignored. ### Why are the changes needed? To make GitHub Actions build stable. ### Does this PR introduce _any_ user-facing change? No, dev-only. ### How was this patch tested? The builds in this PR test it out. Closes #29536 from HyukjinKwon/SPARK-32695. Authored-by: HyukjinKwon <gurwls223@apache.org> Signed-off-by: HyukjinKwon <gurwls223@apache.org> (cherry picked from commit b07e742) Signed-off-by: HyukjinKwon <gurwls223@apache.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
This PR proposes to explicitly cache and hash the files/directories under 'build' for SBT and Zinc at GitHub Actions. Otherwise, it can end up with overwriting
builddirectory. See also #29286 (comment)Previously, other files like
build/mvnandbuild/sbtare also cached and overwritten. So, when you have some changes there, they are ignored.Why are the changes needed?
To make GitHub Actions build stable.
Does this PR introduce any user-facing change?
No, dev-only.
How was this patch tested?
The builds in this PR test it out.