8342988: GHA: Build JTReg in single step

[shipilev]

It is visible in current GHA runs that building jtreg sometimes break on dependencies checkout, for example:

[build.sh][INFO] CYGWIN_OR_MSYS=0
[build.sh][INFO] JAVA_HOME: /usr/lib/jvm/temurin-17-jdk-amd64
[build.sh][INFO] Downloading https://archive.apache.org/dist/ant/binaries/apache-ant-1.10.8-bin.zip to /home/runner/work/jdk/jdk/jtreg/src/make/../build/deps/apache-ant-1.10.8-bin.zip
Error: sh][ERROR] wget exited with exit code 4
Error: Process completed with exit code 1.

I suppose this is a side-effect of pulling lots of dependencies all at once. Building jtreg on every platform exposes us to these issues more often. Additionally, we had problems with building jtreg on various platforms.

We do not have to build jtreg for every platform. We can just have a job step that does it once, and stores it in the cache for all dependent jobs to use. It would also save some build time across various jobs.

---

Progress

• Change must be properly reviewed (1 review required, with at least 1 Reviewer)
• Change must not contain extraneous whitespace
• Commit message must refer to an issue

Issue

• JDK-8342988: GHA: Build JTReg in single step (Enhancement - P4)

Reviewers

• Magnus Ihse Bursie (@magicus - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/21692/head:pull/21692
$ git checkout pull/21692

Update a local copy of the PR:
$ git checkout pull/21692
$ git pull https://git.openjdk.org/jdk.git pull/21692/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 21692

View PR using the GUI difftool:
$ git pr show -t 21692

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/21692.diff

Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back shade! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@shipilev This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8342988: GHA: Build JTReg in single step

Reviewed-by: ihse

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 36 new commits pushed to the master branch:

• 120a935: 8342561: Metaspace for generated reflection classes is no longer needed
• d5fb6b4: 8339939: [JVMCI] Don't compress abstract and interface Klasses
• a5ad974: 8343056: C2: Micro-optimize Node lists grow
• ec06187: 8034066: Incorrect alignment in the "Code" section for "-c -XDdetails" options
• eb3669a: 8340796: Use a consistent order when loading cxq and EntryList
• 0e3fc93: 8342083: Make a few fields in FileSystemPreferences final
• 762a573: 8335880: More troubleshooting tips around windows space in path
• 40e07a7: 8342865: Use type parameter for Class::getPrimitiveClass
• 9e451aa: 8343102: Remove --compress from jlink command lines from jpackage tests
• 873f8a6: 8305406: Add @SPEC tags in java.base/java.* (part 2)
• ... and 26 more: https://git.openjdk.org/jdk/compare/f0b130e54f33d3190640ce33c991e35f27e9f812...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

[openjdk]

@shipilev The following label will be automatically applied to this pull request:

• build

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

[mlbridge]

Webrevs

• 01: Full - Incremental (45288ca4)
• 00: Full (21f65c3f)

[magicus]

Can we trust the cache that much? I mean, up to now it's only been a performance hack, now it will become a necessary part of the pipeline.

[shipilev]

Can we trust the cache that much? I mean, up to now it's only been a performance hack, now it will become a necessary part of the pipeline.

Yeah, I guess that's the risk. I can redo this to use the same upload/download-artifact we use to transfer bundles between build and test jobs. Probably next week.

[shipilev]

Can we trust the cache that much? I mean, up to now it's only been a performance hack, now it will become a necessary part of the pipeline.

Yeah, I guess that's the risk. I can redo this to use the same upload/download-artifact we use to transfer bundles between build and test jobs. Probably next week.

Actually, as I read "Caching dependencies to speed up workflows", this is not as high of the risk: "GitHub will remove any cache entries that have not been accessed in over 7 days. There is no limit on the number of caches you can store, but the total size of all caches in a repository is limited to 10 GB. Once a repository has reached its maximum cache storage, the cache eviction policy will create space by deleting the oldest caches in the repository."

I don't think we are anywhere close to 10GB limit for a single PR. I guess we can have some thrashing when there are multiple PRs in flight. Using artifacts would likely be more bullet-proof.

[shipilev]

Reworked for artifacts. My limited tests passed. Let's see if full GHA test passes with it.

[shipilev]

Reworked for artifacts. My limited tests passed. Let's see if full GHA test passes with it.

Passed. @magicus, take a look again?

[magicus]

Overall this looks very nice; thanks! I left a few qestions, just.

[magicus]

Ship it!

[shipilev]

Aye.

/integrate

[openjdk]

Going to push as commit 88dc655.
Since your change was applied there have been 49 commits pushed to the master branch:

• df3473e: 8343178: Test BasicTest.java javac compile fails cannot find symbol
• 54327bc: 8342962: [s390x] TestOSRLotsOfLocals.java crashes
• f0075d5: 8343115: SkipIfEqual class is not used after JDK-8335946
• 90bd544: 8342958: Use jvmArgs consistently in microbenchmarks
• d49f210: 8342040: Further improve entry lookup performance for multi-release JARs
• d2e716e: 8331958: Update PC/SC Lite for Suse Linux to 2.3.0
• a95374f: 8343101: Rework BasicTest.testTemp test cases
• 00fe9f7: 8343100: Consolidate EmptyFolderTest and EmptyFolderPackageTest jpackage tests into single java file
• 9f6d5b4: 8343020: (fs) Add support for SecureDirectoryStream on macOS
• 1341b81: 8341666: FileInputStream doesn't support readAllBytes() or readNBytes(int) on pseudo devices
• ... and 39 more: https://git.openjdk.org/jdk/compare/f0b130e54f33d3190640ce33c991e35f27e9f812...master

Your commit was automatically rebased without conflicts.

[openjdk]

@shipilev Pushed as commit 88dc655.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.