Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for .jruby.java_opts home and cwd files #5826

Merged
merged 14 commits into from Sep 19, 2019

Conversation

@headius
Copy link
Member

headius commented Aug 9, 2019

This implements the dotfile idea from #5824 in our bash launcher. The order of precedence for options, indicated by their position in the java command line, goes as follows:

  • home dir .jruby.java_opts
  • current dir .jruby.java_opts
  • JAVA_OPTS environment variable
  • command line flags

This allows users to set up global, local, and environment-based java options but still override those at the command line.

An equivalent change will need to go into jruby-launcher.

This implements the dotfile idea from #5824 in our bash launcher.
The order of precedence for options, indicated by their position
in the java command line, goes as follows:

* home dir .jruby.java_opts
* current dir .jruby.java_opts
* JAVA_OPTS environment variable
* command line flags

This allows users to set up global, local, and environment-based
java options but still override those at the command line.

An equivalent change will need to go into jruby-launcher.
headius added 2 commits Aug 18, 2019
* Only use @file syntax on Java 9+.
* Java 8 expands .java_opts files directly on the command line.
* Add a default bin/.dev_mode.jruby_opts with our --dev flags.
  This will allow users or distributors of JRuby to customize
  these flags for a particular installation, for example by adding
  flags for OpenJ9.
* Add support for bin/.jruby.java_opts for installation-specific
  always-on options.
* Re-ordered some logic.
* Set JAVA_HOME to provide better detection of Java 9.
@headius

This comment has been minimized.

Copy link
Member Author

headius commented Aug 18, 2019

Most recent commit adds a bit more logic:

  • The flags for --dev are now pulled from bin/.dev_mode.java_opts.
  • A "local" .jruby_opts file will be loaded from bin/.jruby.java_opts.

It also turns out that this feature was added in Java 9, so I modified the script to source the related files directly into the command line when we are not running on Java 9 or higher.

The commit also contains logic to set JAVA_HOME if it is not set, based on the result of which $JAVACMD or which java. This is used to detect Java 9+'s jmods directory.

headius added 4 commits Aug 18, 2019
@headius

This comment has been minimized.

Copy link
Member Author

headius commented Aug 19, 2019

Here's a page documenting this feature. It can move into the wiki once this is accepted.

Passing JVM options to JRuby's launcher

JRuby is a JVM language, and that means you will frequently want to pass JVM options for tuning the garbage collector or JIT, opening up classes and modules, or logging and profiling. If you want to have these options passed to every JRuby run, you can either put them in the JAVA_OPTS environment variable, or use JRuby-specific .jruby.java_opts files.

Command-line Argument Files

Java 9 introduced support for command-line argument files, which allows you to aggregate JVM options in a file and include those options on the command line with the syntax @<filename> rather than passing them directly. JRuby builds on this feature and supports it on Java 8 via our .jruby.java_opts files.

.jruby.java_opts

The format of .jruby.java_opts is as documented above, with options separated by whitespace (spaces, tabs, newlines, etc) or enclosed in quotes to preserve spaces.

On startup, JRuby will look for .jruby.java_opts files in the following locations:

  1. In the installed JRuby's bin/.jruby.java_opts (empty by default).
  2. In the current user's home directory at ~/.jruby.java_opts.
  3. In the current working directory at ./.jruby.java_opts.

Options passed through JRuby to the JVM via different mechanisms will override each other. Given options are processed in the following order, with later options overriding earlier ones:

  1. Any .jruby.java_opts files will be processed in the order shown above.
  2. The JAVA_OPTS environment variable will be processed.
  3. Any flags passed at JRuby's command line using -J<option> will be processed last.

An example file, used by JRuby's --dev flag, can be found in the JRuby install under bin/.dev_mode.java_opts.

Notes

  • Argument files are only directly supported by the java launcher in Java 9 and higher, but JRuby also supports using .jruby.java_opts on Java 8. Because we do this by expanding the file's contents directly into our final java command, it's possible for this argument list to exceed some shells' character limit.
  • There's no way at present to separate .jruby.java_opts files by Java version or JVM implementation, mostly due to the difficulty of detecting what JVM you are using before startup. We are exploring other options here.
  • JRuby also support passing JRuby options via .jrubyrc files from the user's home or from the current directory. See Configuring JRuby for more information on .jrubyrc files.
  • A separate options file, in bin/.jruby.module_opts, is used when running on a Java Module-enabled JVM. By default it opens packages that JRuby needs for Ruby compatibility.
@headius

This comment has been minimized.

Copy link
Member Author

headius commented Aug 19, 2019

@enebo raised a concern about how we'll know where options are coming from with all these options, which was really a valid concern before but moreso with .java_opts files. I'm going to look at adding a --environment flag of some kind that basically dumps out all the magic we do in the launcher.

This also needs to be matched in the C++ launcher. That will be a bit of work.

@headius

This comment has been minimized.

Copy link
Member Author

headius commented Aug 19, 2019

Here's a quick prototype output from --environment. Adding the --environment flag will print out this info and not actually run JRuby.

[] ~/projects/jruby $ jruby.bash --dev --environment
JRuby Environment:
JRUBY_HOME resolved to /Users/headius/projects/jruby based on executable /Users/headius/projects/jruby/bin/jruby.bash
JAVACMD resolved to /Library/Java/JavaVirtualMachines/jdk-11.0.2+9-hotspot/Contents/Home/bin/java
JAVA_HOME resolved to /Library/Java/JavaVirtualMachines/jdk-11.0.2+9-hotspot/Contents/Home
Java modules detected at /Library/Java/JavaVirtualMachines/jdk-11.0.2+9-hotspot/Contents/Home/jmods
Including Java options from /Users/headius/projects/jruby/bin/.jruby.java_opts
Including Java options from /Users/headius/projects/jruby/.jruby.java_opts
Setting Java stack size with -Xss2048k
Command line options: --dev
Including dev mode options from /Users/headius/projects/jruby/bin/.dev_mode.java_opts
Full Java command line: /Library/Java/JavaVirtualMachines/jdk-11.0.2+9-hotspot/Contents/Home/bin/java @/Users/headius/projects/jruby/bin/.jruby.java_opts @/Users/headius/projects/jruby/.jruby.java_opts @/Users/headius/projects/jruby/bin/.dev_mode.java_opts -Xss2048k --add-opens java.base/java.io=org.jruby.dist --add-opens java.base/java.nio.channels=org.jruby.dist --add-opens java.base/sun.nio.ch=org.jruby.dist -Djffi.boot.library.path=/Users/headius/projects/jruby/lib/jni -Dfile.encoding=UTF-8 -Djava.security.egd=file:/dev/urandom --module-path /Users/headius/projects/jruby/lib/jruby.jar -classpath : -Djruby.home=/Users/headius/projects/jruby -Djruby.lib=/Users/headius/projects/jruby/lib -Djruby.script=jruby -Djruby.shell=/bin/sh org.jruby.Main
@headius

This comment has been minimized.

Copy link
Member Author

headius commented Aug 19, 2019

Slightly less verbose version of output:

JRuby Environment
=================
JRuby executable: /Users/headius/projects/jruby/bin/jruby.bash
JRUBY_HOME: /Users/headius/projects/jruby
JAVACMD: /Library/Java/JavaVirtualMachines/jdk-11.0.2+9-hotspot/Contents/Home/bin/java
JAVA_HOME: /Library/Java/JavaVirtualMachines/jdk-11.0.2+9-hotspot/Contents/Home
Java modules detected
Adding Java options from: /Users/headius/projects/jruby/bin/.jruby.java_opts
Adding Java options from: /Users/headius/projects/jruby/.jruby.java_opts
JAVA_STACK: -Xss2048k
JRuby command line options: --dev
Adding Java options from: /Users/headius/projects/jruby/bin/.dev_mode.java_opts
Full Java command line: /Library/Java/JavaVirtualMachines/jdk-11.0.2+9-hotspot/Contents/Home/bin/java @/Users/headius/projects/jruby/bin/.jruby.java_opts @/Users/headius/projects/jruby/.jruby.java_opts @/Users/headius/projects/jruby/bin/.dev_mode.java_opts -Xss2048k --add-opens java.base/java.io=org.jruby.dist --add-opens java.base/java.nio.channels=org.jruby.dist --add-opens java.base/sun.nio.ch=org.jruby.dist -Djffi.boot.library.path=/Users/headius/projects/jruby/lib/jni -Dfile.encoding=UTF-8 -Djava.security.egd=file:/dev/urandom --module-path /Users/headius/projects/jruby/lib/jruby.jar -classpath : -Djruby.home=/Users/headius/projects/jruby -Djruby.lib=/Users/headius/projects/jruby/lib -Djruby.script=jruby -Djruby.shell=/bin/sh org.jruby.Main

Note that the native launcher does have a verbose mode that outputs some of this information.

headius added 2 commits Aug 19, 2019
This flag gathers relevant environment info from environment
variables, JVM argument files, and command line flags to produce
a log of every config used to boot the JVM for JRuby.

Note this does not currently include .jrubyrc contents because
that file is processed after JVM startup and is only used for
JRuby system properties. Support could be added but would
duplicate the logic we do at runtime (unless we also move that
logic to the launchers).
* Move module opens to bin/.jruby.module_opts and add to log
* Move JRuby options log up
* Remove JAVA_STACK since it will be in final Java options log
* Add --environment to --help output
* Minor wording tweaks
@headius

This comment has been minimized.

Copy link
Member Author

headius commented Aug 19, 2019

Final output pending input from others:

$ jruby.bash --environment -e 1
JRuby Environment
=================
JRuby executable: /Users/headius/projects/jruby/bin/jruby.bash
JRuby command line options: --environment -e 1
JRUBY_HOME: /Users/headius/projects/jruby
JRUBY_OPTS: 
JAVA_OPTS: 
JAVACMD: /Library/Java/JavaVirtualMachines/jdk-11.0.2+9-hotspot/Contents/Home/bin/java
JAVA_HOME: /Library/Java/JavaVirtualMachines/jdk-11.0.2+9-hotspot/Contents/Home
Adding Java options from: /Users/headius/projects/jruby/bin/.jruby.java_opts
Adding Java options from: /Users/headius/projects/jruby/.jruby.java_opts
Adding module flags from: /Users/headius/projects/jruby/bin/.jruby.module_opts
Java command line: /Library/Java/JavaVirtualMachines/jdk-11.0.2+9-hotspot/Contents/Home/bin/java @/Users/headius/projects/jruby/bin/.jruby.java_opts @/Users/headius/projects/jruby/.jruby.java_opts -Xss2048k @/Users/headius/projects/jruby/bin/.jruby.module_opts -Djffi.boot.library.path=/Users/headius/projects/jruby/lib/jni -Dfile.encoding=UTF-8 -Djava.security.egd=file:/dev/urandom --module-path /Users/headius/projects/jruby/lib/jruby.jar -classpath : -Djruby.home=/Users/headius/projects/jruby -Djruby.lib=/Users/headius/projects/jruby/lib -Djruby.script=jruby -Djruby.shell=/bin/sh org.jruby.Main -e 1
@headius

This comment has been minimized.

Copy link
Member Author

headius commented Aug 19, 2019

Wiki page content for --environment

Inspect JRuby's Environment

JRuby's command-line launcher provides a --environment flag you can use to print out information about the JVM, environment variables, and JRuby installation that we use to launch JRuby.

Example output:

$ jruby.bash --environment -e 1
JRuby Environment
=================
JRuby executable: /Users/headius/projects/jruby/bin/jruby.bash
JRuby command line options: --environment -e 1
JRUBY_HOME: /Users/headius/projects/jruby
JRUBY_OPTS: 
JAVA_OPTS: 
JAVACMD: /Library/Java/JavaVirtualMachines/jdk-11.0.2+9-hotspot/Contents/Home/bin/java
JAVA_HOME: /Library/Java/JavaVirtualMachines/jdk-11.0.2+9-hotspot/Contents/Home
Adding Java options from: /Users/headius/projects/jruby/bin/.jruby.java_opts
Adding Java options from: /Users/headius/projects/jruby/.jruby.java_opts
Adding module flags from: /Users/headius/projects/jruby/bin/.jruby.module_opts
Java command line: /Library/Java/JavaVirtualMachines/jdk-11.0.2+9-hotspot/Contents/Home/bin/java @/Users/headius/projects/jruby/bin/.jruby.java_opts @/Users/headius/projects/jruby/.jruby.java_opts -Xss2048k @/Users/headius/projects/jruby/bin/.jruby.module_opts -Djffi.boot.library.path=/Users/headius/projects/jruby/lib/jni -Dfile.encoding=UTF-8 -Djava.security.egd=file:/dev/urandom --module-path /Users/headius/projects/jruby/lib/jruby.jar -classpath : -Djruby.home=/Users/headius/projects/jruby -Djruby.lib=/Users/headius/projects/jruby/lib -Djruby.script=jruby -Djruby.shell=/bin/sh org.jruby.Main -e 1

This information may help you figure out configuration problems. We recommend including this output when filing a JRuby issue.

headius added 2 commits Sep 11, 2019
@headius headius force-pushed the headius:dotfile_java_opts branch from 1315ca9 to ebed58d Sep 19, 2019
headius added 2 commits Sep 19, 2019
* Better formatting for environment flag
* Juggled sections around to keep relevent bits closer together
* All options files use the same function and formatting
* Options from options files are also printed out
* Removed no-longer-necessary MacOS file.encoding hack
* Removed JAVA_VM env logic for -client vs -server
@headius headius merged commit 76f0f51 into jruby:master Sep 19, 2019
4 of 5 checks passed
4 of 5 checks passed
continuous-integration/travis-ci/pr The Travis CI build is in progress
Details
jruby.jruby Build #20190919.5 succeeded
Details
jruby.jruby (Job linux) Job linux succeeded
Details
jruby.jruby (Job mac) Job mac succeeded
Details
jruby.jruby (Job windows) Job windows succeeded
Details
@enebo enebo added this to the JRuby 9.2.9.0 milestone Oct 30, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants
You can’t perform that action at this time.