Skip to content

8294941: GHA: Cut down cross-compilation sysroots #10601

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

shipilev
Copy link
Member

@shipilev shipilev commented Oct 7, 2022

In GHA cross-compilation, we don't need everything from the sysroot, because we use native compilers for the compilation. Therefore, we can cut them even deeper and reclaim significant amount of cache space. This would also make cache eviction less likely, which saves time on recreating the sysroots when that happens.

Current sysroots take about 300M per arch, which quickly multiplies up for multiple arches (we build 5 now), users (ballparking at about 200 active contributors). This amounts to about 1500M caches per job, and 300G in caches across our org for a single JDK tree. Update releases also use GHA for their testing, but the number of contributors there is lower. Still, I think it slides to TB territory now.

In my local tests, new aarch64 sysroot tar.gz gets reduced from 315M to just 85M, about 3.7x improvement. Even deeper cuts are possible, but they would affect the redundant .so-s in /usr/lib/${arch}/. That would be much more fragile to do and might affect our builds. Therefore, I stopped at doing just these.


Progress

  • Change must be properly reviewed (1 review required, with at least 1 Reviewer)
  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue

Issue

  • JDK-8294941: GHA: Cut down cross-compilation sysroots

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk pull/10601/head:pull/10601
$ git checkout pull/10601

Update a local copy of the PR:
$ git checkout pull/10601
$ git pull https://git.openjdk.org/jdk pull/10601/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 10601

View PR using the GUI difftool:
$ git pr show -t 10601

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/10601.diff

@bridgekeeper
Copy link

bridgekeeper bot commented Oct 7, 2022

👋 Welcome back shade! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk openjdk bot added the rfr Pull request is ready for review label Oct 7, 2022
@openjdk
Copy link

openjdk bot commented Oct 7, 2022

@shipilev The following label will be automatically applied to this pull request:

  • build

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added the build build-dev@openjdk.org label Oct 7, 2022
@mlbridge
Copy link

mlbridge bot commented Oct 7, 2022

Webrevs

Copy link
Contributor

@RealCLanger RealCLanger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

@openjdk
Copy link

openjdk bot commented Oct 7, 2022

@shipilev This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8294941: GHA: Cut down cross-compilation sysroots

Reviewed-by: clanger, erikj

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 17 new commits pushed to the master branch:

  • 67210ab: 8286394: Address possibly lossy conversions in jdk.naming.dns
  • 7a194d3: 8290154: [JVMCI] partially implement JVMCI for RISC-V
  • b38bed6: 8294308: Allow dynamically choosing the MEMFLAGS of a type without ResourceObj
  • 118d93b: 8294907: Remove unused NativeLookup::dll_load
  • 1fda842: 8291429: java/lang/Thread/virtual/ThreadAPI.java timed out on single core system
  • 2d25c0a: 8292280: Unused field 'keyListener' in BasicRadioButtonUI
  • 0ad6803: 8293810: Remove granting of RuntimePermission("stopThread") from tests
  • cf84c8e: 8292975: javac produces code that crashes with LambdaConversionException
  • f3a44a4: 8075916: The regression-swing case failed as colored text is not shown on disabled checkbox and radio button with Nimbus LAF
  • 37bd4fb: 6852577: Only for Nimbus LAF UIManager.get("PasswordField.echoChar") is null
  • ... and 7 more: https://git.openjdk.org/jdk/compare/2ceebf681ffd6f9bf6967fd81b30d721bc010b94...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready Pull request is ready to be integrated label Oct 7, 2022
@shipilev
Copy link
Member Author

shipilev commented Oct 7, 2022

Thanks! If any build-dev folks ack this, I can integrate this today :)

Copy link
Member

@erikj79 erikj79 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems fine to me. Nice space savings!

@shipilev
Copy link
Member Author

shipilev commented Oct 7, 2022

Thanks!

/integrate

@openjdk
Copy link

openjdk bot commented Oct 7, 2022

Going to push as commit a1747ef.
Since your change was applied there have been 17 commits pushed to the master branch:

  • 67210ab: 8286394: Address possibly lossy conversions in jdk.naming.dns
  • 7a194d3: 8290154: [JVMCI] partially implement JVMCI for RISC-V
  • b38bed6: 8294308: Allow dynamically choosing the MEMFLAGS of a type without ResourceObj
  • 118d93b: 8294907: Remove unused NativeLookup::dll_load
  • 1fda842: 8291429: java/lang/Thread/virtual/ThreadAPI.java timed out on single core system
  • 2d25c0a: 8292280: Unused field 'keyListener' in BasicRadioButtonUI
  • 0ad6803: 8293810: Remove granting of RuntimePermission("stopThread") from tests
  • cf84c8e: 8292975: javac produces code that crashes with LambdaConversionException
  • f3a44a4: 8075916: The regression-swing case failed as colored text is not shown on disabled checkbox and radio button with Nimbus LAF
  • 37bd4fb: 6852577: Only for Nimbus LAF UIManager.get("PasswordField.echoChar") is null
  • ... and 7 more: https://git.openjdk.org/jdk/compare/2ceebf681ffd6f9bf6967fd81b30d721bc010b94...master

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot added the integrated Pull request has been integrated label Oct 7, 2022
@openjdk openjdk bot closed this Oct 7, 2022
@openjdk openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Oct 7, 2022
@openjdk
Copy link

openjdk bot commented Oct 7, 2022

@shipilev Pushed as commit a1747ef.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@shipilev shipilev deleted the JDK-8294941-gha-cut-sysroots branch October 21, 2022 09:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build build-dev@openjdk.org integrated Pull request has been integrated
Development

Successfully merging this pull request may close these issues.

3 participants