Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvements for modules on Java 9+ #5363

Merged
merged 2 commits into from Oct 15, 2018

Conversation

Projects
None yet
1 participant
@headius
Copy link
Member

headius commented Oct 12, 2018

This PR will track a few small commits to improve how we start up on Java 9+, with a specific eye for running well on Java 11.

Use --module-path for JRuby when running on a jmods JDK.
By moving JRuby into the module path, we are able to silence
some of the reflection warnings. This will also be necessary
for users that want to expose core JDK classes to JRuby for
compatibility purposes.

The logic here looks in $JAVA_HOME for the 'jmods' dir, which
should be there for all OpenJDK 9+ and not in earlier releases.
If that dir is present, we alter the command line in the follwing
ways:

* If running verified, a la VERIFY_JRUBY=1, all jars including
  JRuby will be loaded via --module-path instead of -classpath.
* If running unverified, a la bootclasspath, the bootclasspath
  flag is replaced by --module-path, and an additional flag
  -verify:none is added to  blunt the cost of booting. This is not
  ideal, but all libraries we ship with and all bytecode we
  generate have been verified many times over in testing.

Note that these only affect launching JRuby from the dist release.
We will need similar changes to the .cmd launcher and the native
jruby-launcher.

@headius headius added this to the JRuby 9.2.1.0 milestone Oct 12, 2018

@headius

This comment has been minimized.

Copy link
Member Author

headius commented Oct 12, 2018

@kares @enebo I'm terrible at [ba]sh...double check my work here. Any concerns, like about -Xverify:none?

@headius

This comment has been minimized.

Copy link
Member Author

headius commented Oct 12, 2018

Ok some thoughts on verification, since I'm reluctant to switch it completely off like this...

First, exposure. There's a number of pages that recommend never turning it off, but as we know lots of folks turn it off in development to speed up commands. This change in the jruby script, however, would also happen in a production setting. You'd have to explicitly turn it back on with VERIFY_JRUBY=1.

I have not researched the likelihood or prior art of exploiting Hotspot with verification off.

The main reason this needs to be done is because the bootclasspath flag and the Java module system do not work together, due to boot classes being able to circumvent some aspects of the module system's security guarantees (I believe this is the concern).

So if we can't turn verification off and can't use bootclasspath, we have two options I see:

  • Use AppCDS to eliminate most of the cost of verification. However this is still a work in progress, both in OpenJDK and in the tool I'm working on for JRuby users: https://github.com/jruby/jruby-startup
  • Always verify.

This last option will slow down startup for most users because they have been running with JRuby in bootclasspath (and therefore unverified) previously.

For jruby -S gem list with just a couple dozen gems:

No flags: 3.0-3.2s
With --dev: 2.18s
Verify all: 3.2-3.6s
Verify all --dev: 2.4s

The JIT results are notable noisy, but overall the verification appears to add from 0.2 to 0.4s to startup.

With AppCDS, no verification, --dev: 1.7s
With AppCDS, full verification, --dev: 1.7-1.8s

So turning verification on may be an acceptable step backwards to help move toward AppCDS use?

Note this all comes into play only on Java 9+. Java 8 will continue to work as it does today.

@headius

This comment has been minimized.

Copy link
Member Author

headius commented Oct 12, 2018

Note that AppCDS for a very large app may only help the first few seconds of startup; code loaded after that will be verified anew. This does not really differ much from today, though, since with bootclasspath we limited the unverified body of code to just what JRuby loaded at boot.

@headius

This comment has been minimized.

Copy link
Member Author

headius commented Oct 15, 2018

Ok after some discussion, we're going with the following changes.

When running on Java 9+ (detected by JAVA_HOME/jmods dir), we will not use the unverified bootclasspath logic in the bash script at all and also make the following tweaks:

  • Switch -classpath to --module-path.
  • If JRUBY_HOME/lib/jruby.jsa exists (or another path specifiable with JRUBY_JSA env) add appropriate AppCDS flags to use it.

This means all Java 9+ will now be fully verified, but in module path. Use AppCDS to eliminate the startup hit of that verification.

@headius

This comment has been minimized.

Copy link
Member Author

headius commented Oct 15, 2018

Gist comparing Java 8 startup versus Java 11 with and without AppCDS: https://gist.github.com/headius/6228000dcad848c137554496940a9c31

@headius headius merged commit 04ecb04 into jruby:master Oct 15, 2018

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

@headius headius deleted the headius:module_path_launcher branch Oct 15, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.