-
Notifications
You must be signed in to change notification settings - Fork 5.8k
8311302: Implement JEP 493: Linking Run-Time Images without JMODs #14787
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
👋 Welcome back sgehwolf! A progress list of the required criteria for merging this PR into |
@jerboaa This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration! |
Please keep open, bot. |
a001d8c
to
baeaaf5
Compare
@jerboaa Please do not rebase or force-push to an active PR as it invalidates existing review comments. Note for future reference, the bots always squash all changes into a single commit automatically as part of the integration. See OpenJDK Developers’ Guide for more information. |
This is configurable so add tests for both scenarios.
This uses a stamp file in 'lib/modules' jimage in order to recognize multi-hop links. However, this has the caveat of no longer producing reproducible builds as compared to a packaged module-based link. Add an option to allow for multi-hop, which disables the stamp file addition and makes it reproducible again.
/csr needed |
@jerboaa has indicated that a compatibility and specification (CSR) request is needed for this pull request. @jerboaa please create a CSR request for issue JDK-8311302 with the correct fix version. This pull request cannot be integrated until the CSR request is approved. |
@AlanBateman Hi! Moving the discusion to this PR now. I've updated this patch to do single-hops only by default now. Looks like this:
I'm not particularly fond of that, though, as there is a need to know what is single hop is and what not. My approach is adding a 0-sized stamp file to the Also, if a file is modified the link fails unless it's turned off with an option. Looks like this:
Both things will have CLI options to a) make the exit on mod a warning only and b) allow multi-hop - some tests needed that. Hence, I've marked this PR as needing a CSR for now. More thoughts? |
@AlanBateman Gentle ping. |
On my list, it's a lot to get through and a number of aspects to this that I think will require refinement and discussion. |
Thanks for the heads-up! Your input is much appreciated. |
I did a pass over this to see where this proposal is currently at. At a high-level I think good progress since the discussion on leyden-dev some time ago. A few comments this from this pass: If I read it correctly, the default behavior is now to fail if jlink is executed from a run-time image that has been modified from the original and the packaged modules are not observable on the module path (this includes the default module path effectively appended by jlink). That seems the right policy as it avoids modifications by plugins or conf changes silently propagating. If I read the code correctly, the error when a hash doesn't match is a very terse "Run image links only allow single-hop" so that probably needs to be looked to make sure the message says that the run-time image has been modified and maybe include one of the files/resources that has changed so there is something real to go with the error. The command line options to override and change the error to a warning or silently ignore seems to be "-run-time-ignore-single-hop" and "--run-image-only-warnings". Maybe it should be reduced to just one option and maybe it should contain the word "clone" to something that makes it more obvious that it's just copying or cloning the contents of the modules in the run-time image (I suspect "single hop" won't be understood). Adding a new jdk.tools.jlink.internal.Archive implementation where the resources come from the run-time image seems a sensible approach. I don't particularly like the name "JmodLessArchive" because the original module may not have been packaged as a JMOD file, it may have been a modular JAR. Same comment on the BOM file "jmod_resources" added to the top-level directory of each module. I think it's clever to add this to each module in the container image but part of me wonders if it should be hidden in some way to avoid tools depending on it. I don't have a concrete suggestion except to wonder if jrtfs should filter it out, same thing in the SystemModuleReader code as ModuleReader.find("jmod_resources") will find the resource, could be an attractive nuisance. |
Thanks, Alan!
Correct.
Not quite. See the example I've added in the previous comment. It already includes that info. It looks like this:
The message can of course be modified what we think is best.
I believe keeping them both makes sense. Or at least make it so that it takes an argument to be able to distinguish between the two. The reason why I think this is useful is in the container case, where say - the JDK got installed by an installer - which didn't include packaged modules, but is otherwise a full JDK (with all modules). The installer already does integrity checking of the files and it might be desirable to do multi-hop evolutions, while still wanting to get warnings/errors on modified configuration files, for example. Happy to change the name of that flag/flags to
OK. How about
That seems fine. I can add code to filter those resources out. On the other hand, no special handling is being made for |
The test doesn't need the default module path and default jmods generated from the helper. Using this approach makes it work for linkable run-time images as well.
ACK. I'll do another pass on the changes in the PR. |
Similar to GenerateJLIClassesPluginTest.java, the default module path doesn't need to be explicitly specified.
It's no longer used in any tests.
Done in the latest update. Thanks again for the review! |
GHA failure on aarch64 MacOSX is infra related. |
GHA failure of MacOSX on aarch64 is still infra related. |
I'm worked through the make + src changes and don't have any issues. I skimmed tests some didn't study closely. You will need to bump the copyright header of several jlink src files before integrating. |
Thanks for the review!
OK. Let me work on that. |
Thanks for the update and skimmed through the tests. Looks good. I also agree that better to fail early when detecting patched modules as described in JDK-8343839. |
Thanks for the reviews! I'll integrate this later today. |
/integrate |
Going to push as commit 2ec3580.
Your commit was automatically rebased without conflicts. |
Please review this patch which adds a jlink mode to the JDK which doesn't need the packaged modules being present. A.k.a run-time image based jlink. Fundamentally this patch adds an option to use
jlink
even though your JDK install might not come with the packaged modules (directoryjmods
). This is particularly useful to further reduce the size of a jlinked runtime. After the removal of the concept of a JRE, a common distribution mechanism is still the full JDK with all modules and packaged modules. However, packaged modules can incur an additional size tax. For example in a container scenario it could be useful to have a base JDK container including all modules, but without also delivering the packaged modules. This comes at a size advantage of~25%
. Such a base JDK container could then be used tojlink
application specific runtimes, further reducing the size of the application runtime image (App + JDK runtime; as a single image or separate bundles, depending on the app being modularized).The basic design of this approach is to add a jlink plugin for tracking non-class and non-resource files of a JDK install. I.e. files which aren't present in the jimage (
lib/modules
). This enables producing aJRTArchive
class which has all the info of what constitutes the final jlinked runtime.Basic usage example:
One nice property of this patch is that it can produce an identical (as in binary identical)
java.se
JDK image in run-time image based link mode as compared to a regular jlink with packaged modules present. This has been asserted in newly added tests. In order to prevent accidental copy of modified files in the base JDK a checksum mechanism is in place to fail the link if a runtime image file has been modified. This is also asserted in tests.One limitation of this mode is that there is no way to use it for cross-linking (at least currently). That is, a run-time image based
jlink
needs to happen on the same runtime platform the resulting image is intended to get deployed on.Testing:
jdk/tools/jlink
tests and JDKs built with--enable-linkable-runtime
). See for example the latest run and here.jdk/tools/jlink
tests and JDKs built with--enable-linkable-runtime --enable-keep-packaged-modules
). See for example the latest run and here.jdk/tools/jlink
tests for the default build (--disable-linkable-runtime
), which also runs the tests inruntimeImage
sub-folder here by first creating a run-time image link capable JDK includingjdk.jlink
module.Thoughts?
Progress
Issues
Reviewers
Contributors
<mchung@openjdk.org>
Reviewing
Using
git
Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/14787/head:pull/14787
$ git checkout pull/14787
Update a local copy of the PR:
$ git checkout pull/14787
$ git pull https://git.openjdk.org/jdk.git pull/14787/head
Using Skara CLI tools
Checkout this PR locally:
$ git pr checkout 14787
View PR using the GUI difftool:
$ git pr show -t 14787
Using diff file
Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/14787.diff
Using Webrev
Link to Webrev Comment