Set the jvm arguments of the gradle compiler deamon

[blackdrag]

I am trying to get the Groovy project compiling using Gradle 3.2.1 under jigsaw 155. This time I got blocked by the Gradle Compiler Daemon trying to handle the build scripts. I know how to fix the problem (by adding exports as jvm argument), but I have not found a way to set the jvm arguments of the Gradle Compiler Daemon worker. Here is at least documentation missing. A possible workaround would be to force the build source compilation be part of the same JVM as gradle itself or anything that let´s me set options via GRADLE_OPTS. But I did not find any information on if that is possible either.

[eriwen]

@ghale IIRC there was recent work in this area, correct? Do you have any information for @blackdrag here?

[ghale]

You should be able to do this by setting the fork options on the compile task. For instance:

compileJava {
  options.forkOptions.jvmArgs += "-foo"
}

This is documented through the DSL reference (https://docs.gradle.org/current/dsl/org.gradle.api.tasks.compile.JavaCompile.html) although it would probably require some digging.

[eriwen]

Right, thanks Gary. For some reason I had it in my head that that didn't affect the compiler daemons.

Closing as answered. Please reopen @blackdrag if anything is amiss.

[blackdrag]

I don´t think I have the access rights to reopen the issue. Because I already had this before I filed this issue:

compileJava {
    dependsOn ensureGrammars
   //    options.fork(memoryMaximumSize: javacMain_mx)
   options.forkOptions.jvmArgs << "---fooo"
}

What I had expected to happen was the build script compiler daemon to fail to start because of the invalid JVM option. That did not happen, nor did the option show up in the log:

08:34:31.953 [DEBUG] [org.gradle.process.internal.worker.child.WorkerProcessClassPathProvider] Using worker process classpath: [/home/blackdrag/.gradle/caches/3.2.1/workerMain/gradle-worker.jar]
08:34:31.967 [INFO] [org.gradle.process.internal.DefaultExecHandle] Starting process 'Gradle Compiler Daemon 1'. Working directory: /home/blackdrag/coding/groovy/commit/buildSrc Command: /opt/jdk9-jigsaw-ea-155/bin/java @/tmp/gradle-worker-classpath1103722571087631810txt -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Compiler Daemon 1'
08:34:31.967 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Environment for process 'Gradle Compiler Daemon 1': {PATH=/opt/jdk8/bin:/home/blackdrag/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games, LC_MEASUREMENT=de_DE.UTF-8, XAUTHORITY=/home/blackdrag/.Xauthority, LC_TELEPHONE=de_DE.UTF-8, XMODIFIERS=@im=ibus, GRADLE_OPTS=--add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED, MANDATORY_PATH=/usr/share/gconf/mate.mandatory.path, GDMSESSION=mate, XDG_DATA_DIRS=/usr/share/mate:/usr/local/share/:/usr/share/, LC_TIME=de_DE.UTF-8, TEXTDOMAINDIR=/usr/share/locale/, GTK_IM_MODULE=ibus, DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-ygCln4ToB4,guid=485233b321c5a4ebe00eb14958b863ce, DEFAULTS_PATH=/usr/share/gconf/mate.default.path, XDG_CURRENT_DESKTOP=MATE, SSH_AGENT_PID=1919, COLORTERM=mate-terminal, QT4_IM_MODULE=xim, LC_PAPER=de_DE.UTF-8, SESSION_MANAGER=local/Gr8:@/tmp/.ICE-unix/1841,unix/Gr8:/tmp/.ICE-unix/1841, LOGNAME=blackdrag, PWD=/home/blackdrag/coding/groovy/commit, LANGUAGE=en_US, WINDOWID=48234533, SHELL=/bin/bash, LESSOPEN=| /usr/bin/lesspipe %s, LC_ADDRESS=de_DE.UTF-8, OLDPWD=/home/blackdrag/coding/groovy/commit, GTK_MODULES=gail:atk-bridge:overlay-scrollbar:canberra-gtk-module, CLUTTER_IM_MODULE=xim, MATE_DESKTOP_SESSION_ID=this-is-deprecated, XDG_SESSION_PATH=/org/freedesktop/DisplayManager/Session0, TEXTDOMAIN=im-config, LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:.tar=01;31:.tgz=01;31:.arj=01;31:.taz=01;31:.lzh=01;31:.lzma=01;31:.tlz=01;31:.txz=01;31:.zip=01;31:.z=01;31:.Z=01;31:.dz=01;31:.gz=01;31:.lz=01;31:.xz=01;31:.bz2=01;31:.bz=01;31:.tbz=01;31:.tbz2=01;31:.tz=01;31:.deb=01;31:.rpm=01;31:.jar=01;31:.war=01;31:.ear=01;31:.sar=01;31:.rar=01;31:.ace=01;31:.zoo=01;31:.cpio=01;31:.7z=01;31:.rz=01;31:.jpg=01;35:.jpeg=01;35:.gif=01;35:.bmp=01;35:.pbm=01;35:.pgm=01;35:.ppm=01;35:.tga=01;35:.xbm=01;35:.xpm=01;35:.tif=01;35:.tiff=01;35:.png=01;35:.svg=01;35:.svgz=01;35:.mng=01;35:.pcx=01;35:.mov=01;35:.mpg=01;35:.mpeg=01;35:.m2v=01;35:.mkv=01;35:.webm=01;35:.ogm=01;35:.mp4=01;35:.m4v=01;35:.mp4v=01;35:.vob=01;35:.qt=01;35:.nuv=01;35:.wmv=01;35:.asf=01;35:.rm=01;35:.rmvb=01;35:.flc=01;35:.avi=01;35:.fli=01;35:.flv=01;35:.gl=01;35:.dl=01;35:.xcf=01;35:.xwd=01;35:.yuv=01;35:.cgm=01;35:.emf=01;35:.axv=01;35:.anx=01;35:.ogv=01;35:.ogx=01;35:.aac=00;36:.au=00;36:.flac=00;36:.mid=00;36:.midi=00;36:.mka=00;36:.mp3=00;36:.mpc=00;36:.ogg=00;36:.ra=00;36:.wav=00;36:.axa=00;36:.oga=00;36:.spx=00;36:.xspf=00;36:, SHLVL=1, LC_IDENTIFICATION=de_DE.UTF-8, LC_MONETARY=de_DE.UTF-8, LESSCLOSE=/usr/bin/lesspipe %s %s, COMPIZ_CONFIG_PROFILE=mate, QT_IM_MODULE=ibus, JAVA_HOME=/opt/jdk8, TERM=xterm, XDG_CONFIG_DIRS=/etc/xdg/xdg-mate:/etc/xdg, GNOME_KEYRING_CONTROL=/run/user/1000/keyring-giXpos, XDG_SEAT_PATH=/org/freedesktop/DisplayManager/Seat0, LANG=en_US.UTF-8, XDG_SESSION_ID=c2, QT_STYLE_OVERRIDE=gtk, DISPLAY=:0.0, LC_NAME=de_DE.UTF-8, GDM_LANG=en_US, XDG_GREETER_DATA_DIR=/var/lib/lightdm-data/blackdrag, GPG_AGENT_INFO=/run/user/1000/keyring-giXpos/gpg:0:1, DESKTOP_SESSION=mate, USER=blackdrag, QT_ACCESSIBILITY=1, LC_NUMERIC=de_DE.UTF-8, SSH_AUTH_SOCK=/run/user/1000/keyring-giXpos/ssh, XDG_SEAT=seat0, QT_QPA_PLATFORMTHEME=appmenu-qt5, XDG_VTNR=7, XDG_RUNTIME_DIR=/run/user/1000, HOME=/home/blackdrag, GNOME_KEYRING_PID=1839}

If this is supposed to work for Gradle 3.2.1, then it has a bug. I just checked against 3.4 and there we have:

08:42:03.902 [DEBUG] [org.gradle.process.internal.worker.child.WorkerProcessClassPathProvider] Using worker process classpath: [/home/blackdrag/.gradle/caches/3.4/workerMain/gradle-worker.jar]
08:42:03.921 [INFO] [org.gradle.process.internal.DefaultExecHandle] Starting process 'Gradle Worker Daemon 1'. Working directory: /home/blackdrag/coding/groovy/commit/buildSrc Command: /opt/jdk9-jigsaw-ea-155/bin/java @/tmp/gradle-worker-classpath7652957163780460905txt -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Worker Daemon 1'
08:42:03.922 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Environment for process 'Gradle Worker Daemon 1': {PATH=/opt/jdk8/bin:/home/blackdrag/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games, LC_MEASUREMENT=de_DE.UTF-8, XAUTHORITY=/home/blackdrag/.Xauthority, LC_TELEPHONE=de_DE.UTF-8, XMODIFIERS=@im=ibus, GRADLE_OPTS=--add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED, MANDATORY_PATH=/usr/share/gconf/mate.mandatory.path, GDMSESSION=mate, XDG_DATA_DIRS=/usr/share/mate:/usr/local/share/:/usr/share/, LC_TIME=de_DE.UTF-8, TEXTDOMAINDIR=/usr/share/locale/, GTK_IM_MODULE=ibus, DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-ygCln4ToB4,guid=485233b321c5a4ebe00eb14958b863ce, DEFAULTS_PATH=/usr/share/gconf/mate.default.path, XDG_CURRENT_DESKTOP=MATE, SSH_AGENT_PID=1919, COLORTERM=mate-terminal, QT4_IM_MODULE=xim, LC_PAPER=de_DE.UTF-8, SESSION_MANAGER=local/Gr8:@/tmp/.ICE-unix/1841,unix/Gr8:/tmp/.ICE-unix/1841, LOGNAME=blackdrag, PWD=/home/blackdrag/coding/groovy/commit, LANGUAGE=en_US, WINDOWID=48234533, SHELL=/bin/bash, LESSOPEN=| /usr/bin/lesspipe %s, LC_ADDRESS=de_DE.UTF-8, OLDPWD=/home/blackdrag/coding/groovy/commit, GTK_MODULES=gail:atk-bridge:overlay-scrollbar:canberra-gtk-module, CLUTTER_IM_MODULE=xim, MATE_DESKTOP_SESSION_ID=this-is-deprecated, XDG_SESSION_PATH=/org/freedesktop/DisplayManager/Session0, TEXTDOMAIN=im-config, LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:.tar=01;31:.tgz=01;31:.arj=01;31:.taz=01;31:.lzh=01;31:.lzma=01;31:.tlz=01;31:.txz=01;31:.zip=01;31:.z=01;31:.Z=01;31:.dz=01;31:.gz=01;31:.lz=01;31:.xz=01;31:.bz2=01;31:.bz=01;31:.tbz=01;31:.tbz2=01;31:.tz=01;31:.deb=01;31:.rpm=01;31:.jar=01;31:.war=01;31:.ear=01;31:.sar=01;31:.rar=01;31:.ace=01;31:.zoo=01;31:.cpio=01;31:.7z=01;31:.rz=01;31:.jpg=01;35:.jpeg=01;35:.gif=01;35:.bmp=01;35:.pbm=01;35:.pgm=01;35:.ppm=01;35:.tga=01;35:.xbm=01;35:.xpm=01;35:.tif=01;35:.tiff=01;35:.png=01;35:.svg=01;35:.svgz=01;35:.mng=01;35:.pcx=01;35:.mov=01;35:.mpg=01;35:.mpeg=01;35:.m2v=01;35:.mkv=01;35:.webm=01;35:.ogm=01;35:.mp4=01;35:.m4v=01;35:.mp4v=01;35:.vob=01;35:.qt=01;35:.nuv=01;35:.wmv=01;35:.asf=01;35:.rm=01;35:.rmvb=01;35:.flc=01;35:.avi=01;35:.fli=01;35:.flv=01;35:.gl=01;35:.dl=01;35:.xcf=01;35:.xwd=01;35:.yuv=01;35:.cgm=01;35:.emf=01;35:.axv=01;35:.anx=01;35:.ogv=01;35:.ogx=01;35:.aac=00;36:.au=00;36:.flac=00;36:.mid=00;36:.midi=00;36:.mka=00;36:.mp3=00;36:.mpc=00;36:.ogg=00;36:.ra=00;36:.wav=00;36:.axa=00;36:.oga=00;36:.spx=00;36:.xspf=00;36:, SHLVL=1, LC_IDENTIFICATION=de_DE.UTF-8, LC_MONETARY=de_DE.UTF-8, LESSCLOSE=/usr/bin/lesspipe %s %s, COMPIZ_CONFIG_PROFILE=mate, QT_IM_MODULE=ibus, JAVA_HOME=/opt/jdk8, TERM=xterm, XDG_CONFIG_DIRS=/etc/xdg/xdg-mate:/etc/xdg, GNOME_KEYRING_CONTROL=/run/user/1000/keyring-giXpos, XDG_SEAT_PATH=/org/freedesktop/DisplayManager/Seat0, LANG=en_US.UTF-8, XDG_SESSION_ID=c2, QT_STYLE_OVERRIDE=gtk, DISPLAY=:0.0, LC_NAME=de_DE.UTF-8, GDM_LANG=en_US, XDG_GREETER_DATA_DIR=/var/lib/lightdm-data/blackdrag, GPG_AGENT_INFO=/run/user/1000/keyring-giXpos/gpg:0:1, DESKTOP_SESSION=mate, USER=blackdrag, QT_ACCESSIBILITY=1, LC_NUMERIC=de_DE.UTF-8, SSH_AUTH_SOCK=/run/user/1000/keyring-giXpos/ssh, XDG_SEAT=seat0, QT_QPA_PLATFORMTHEME=appmenu-qt5, XDG_VTNR=7, XDG_RUNTIME_DIR=/run/user/1000, HOME=/home/blackdrag, GNOME_KEYRING_PID=1839}

Which does not look all that different to me and still fails to include my JVM option. It does not even include GRADLE_OPTS. I know the option works for forked javac´s, but the build source is different. And if this is supposed to be the right way, to do it, then consider this a bug report, because it does not work, and blocks me from using JAVA9 with jigsaw

[blackdrag]

one more thing. On github I can reopen an issue only if I closed it. Since @eriwen closed this issue I cannot reopen it

[ghale]

@blackdrag I assume this is for groovy-core? I added exactly the configuration you have above to groovy-core/build.gradle and I got a failure to start the compiler daemon. This happened for the version of gradle that was in the wrapper (2.3), 3.2.1 and 3.4. The process that was started had the added option:

10:37:19.096 [INFO] [org.gradle.process.internal.DefaultExecHandle] Starting process 'Gradle Compiler Daemon 1'. Working directory: /Users/ghale/repos/groovy-core Command: /Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/Home/bin/java ---fooo -Xmx384m -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -cp /Users/ghale/.gradle/wrapper/dists/gradle-2.3-bin/7a1ngj6cc7aafikc3uq95v6rn0/gradle-2.3/lib/gradle-base-services-2.3.jar:/Users/ghale/.gradle/wrapper/dists/gradle-2.3-bin/7a1ngj6cc7aafikc3uq95v6rn0/gradle-2.3/lib/gradle-core-2.3.jar:/Users/ghale/.gradle/wrapper/dists/gradle-2.3-bin/7a1ngj6cc7aafikc3uq95v6rn0/gradle-2.3/lib/gradle-cli-2.3.jar:/Users/ghale/.gradle/wrapper/dists/gradle-2.3-bin/7a1ngj6cc7aafikc3uq95v6rn0/gradle-2.3/lib/gradle-native-2.3.jar:/Users/ghale/.gradle/wrapper/dists/gradle-2.3-bin/7a1ngj6cc7aafikc3uq95v6rn0/gradle-2.3/lib/gradle-messaging-2.3.jar:/Users/ghale/.gradle/wrapper/dists/gradle-2.3-bin/7a1ngj6cc7aafikc3uq95v6rn0/gradle-2.3/lib/slf4j-api-1.7.7.jar:/Users/ghale/.gradle/wrapper/dists/gradle-2.3-bin/7a1ngj6cc7aafikc3uq95v6rn0/gradle-2.3/lib/logback-classic-1.0.13.jar:/Users/ghale/.gradle/wrapper/dists/gradle-2.3-bin/7a1ngj6cc7aafikc3uq95v6rn0/gradle-2.3/lib/logback-core-1.0.13.jar:/Users/ghale/.gradle/wrapper/dists/gradle-2.3-bin/7a1ngj6cc7aafikc3uq95v6rn0/gradle-2.3/lib/jul-to-slf4j-1.7.7.jar:/Users/ghale/.gradle/wrapper/dists/gradle-2.3-bin/7a1ngj6cc7aafikc3uq95v6rn0/gradle-2.3/lib/guava-jdk5-17.0.jar org.gradle.process.internal.launcher.GradleWorkerMain 'Gradle Compiler Daemon 1'

I'm guessing something else is going on here (possibly something overwriting jvmArgs later in build script?). Is your not-working configuration on a branch or somewhere that I could take a look at it?

[ghale]

OK, actually I just took a closer look at your code snippet above and I think I know what the problem may be. You have the following commented out:

//    options.fork(memoryMaximumSize: javacMain_mx)

options.fork() actually does two things. It sets the provided mapped properties in options.forkOptions, but it also sets the options.fork property's value to true (it's false by default). When this is commented out, it runs the compiler in-process and the fork options don't have an effect. If you either uncomment this line or add options.fork = true you should see the option added to the compiler daemon.

[blackdrag]

This is for groovy-core, yes, and I will try soon. But could you explain "in process" a bit more? Because it is not the process I start with the gradle command, that I can influence using GRADLE_OPTS. Why is gradle creating separate process for the builder daemon, but lets you set options for it only in fork mode?

[blackdrag]

ok, after trying out for a few minutes... Is it possible this behaves different on jdk8 and jdk9? Because my "---fooo" seems not to appear using:

Java(TM) SE Runtime Environment (build 9-ea+155-jigsaw-nightly-h6059-20170208)
Java HotSpot(TM) 64-Bit Server VM (build 9-ea+155-jigsaw-nightly-h6059-20170208, mixed mode)

[blackdrag]

an I did just see I still have artefacts from several tries before... So I also have

org.gradle.jvmargs=-ea -Xmx1G --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED
#org.gradle.jvmargs=-Xmx1G -XX:MaxPermSize=384m -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled

in gradle.settings and also org.gradle.daemon=false. But as you can see setting the daemon to false does not prevent the creation of the builder daemon. Which leads me back to not understanding the handling of this build script daemon. I mean why is a fork option on compileJava controlling (yet to be verified) the buildscript daemon? Why can`t I disabled the usage of daemons with org.gradle.daemon=false anymore? why does the build script daemon ignore org.gradle.jvmargs? Why does the daemon not take GRADLE_OPTS?

You see I am hopelessly lost and I am probably circling around a bit because I have no clue about how things are working anymore and it would help understanding how things are supposed to be working at least. At this point I can´t even tell if the forked JVMs for compileGroovy or compileJava have anything to do with the one for the build script. My assumption would have been that I can set these separately, but your suggestion is telling me other things. I would have thought for example that we could have the build script compilation even without a java or groovy compiling build. How does it then make sense to use compileJava fork options to control this?

Really, not ranting here, just trying to understand the workings of this build script daemon.

[reinhapa]

I got the same issues here with the latest Java(TM) SE Runtime Environment (build 9-ea+158) preview build.

There is a epic #890 that should add support for JDK 9 also, unfortunately there where no actual updates since January. I would like to help out in testing more in this area, but did not jet receive any response jet.

At last, I saw that you still use the -XX:MaxPermSize=384m parameter. I just wanted to to point out, that this option has been removed since JDK 8, due the fact that Perm memory is no longer available from that JDK on. A similar option would be -XX:MaxMetaspaceSize=384m

[blackdrag]

@reinhapa I actually used only

org.gradle.jvmargs=-ea -Xmx1G --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED

[ghale]

So, a couple of different things going on here and hopefully I can clear some things up. There are two different "daemons" in play here. There is the "Gradle" daemon and then there are compiler daemons.

The Gradle daemon is a long-running process that stays alive after the build is done and maintains all sorts of state about your build so that the next time a build is run, it's warm and everything runs faster. You can think of them as a build process that runs a build but instead of exiting, waits for additional build requests. When you set org.gradle.daemon=false, you're telling Gradle not to run the build using the Gradle daemon and to shut everything down after the build is complete. The compiler daemons are long(-ish) running processes that only handle compilation requests for a particular language (e.g. Java) and are stopped at the end of a build session. They are separate and there is no relationship to Gradle daemon configuration - i.e. org.gradle.daemon has no implications for compiler daemons, and you can use compiler daemons or not regardless of whether you are using the Gradle daemon. Now, org.gradle.jvmargs speaks to what options should be passed to the java runtime when the build jvm is started. When the Gradle daemon is enabled, these are the options used to start its JVM. On the other hand, when the daemon is disabled and org.gradle.jvmargs is set, Gradle has to accommodate those settings if the current JVM is not compatible. So what happens is that it will start a "single use" daemon which starts with the provided arguments, accepts a single build request and then stops at the end of the build. So, although it is starting a "daemon" process, it's really just a way of spawning another build JVM with the requested arguments, and its not actually long-lived.

With respect to the compiler daemons, you can either run them in-process or out-of-process. "In-process" means they run in the build JVM (i.e. the Gradle daemon) and compilation is done using the javax.tools.JavaCompiler api. "Out-of-process" means that Gradle starts a compiler daemon that also uses the JavaCompiler api, but after a compilation it waits for further requests. There are a couple of reasons why you might want to run compilation out-of-process - e.g. to use a different JDK for compilation than Gradle is running with, or to set different system properties, or to set a larger heap size, etc, etc. The options.forkOptions setting on JavaCompile tasks allow you to control the JVM arguments and other aspects of the compiler daemon.

Now, aside from JVM arguments for the compiler daemon itself, there are also compilation arguments which are the arguments to use for a particular compilation task (and which are passed to the underlying JavaCompiler instance). Many of the options on JavaCompile.options set compiler arguments, but you can also add your own options using options.compilerArgs.

I hope this clears some things up. Let me know if anything is not clear.

[sdfelts]

I've been working with JDK9 since the early builds. The restriction on setAccessible was a big step backward for Gradle. While it's possible to add --add-opens to the environment generally, it does open it up for other software beside Gradle. On the other hand, trying to change every project build.gradle to have the right packages opened is very invasive. The solution that I found worked best in our environment was to introduce a shell script to invoke Gradle. In the shell script, when running with JDK9 I export _JAVA_OPTIONS with all of the --add-opens arguments needed for all of the projects in our build. It's also a good place to add -Dsun.reflect.debugModuleAccessChecks=true to track down additional packages that need to be added.
This isolates changes to this shell and the opened packages only to gradle and stuff executed under gradle.

I was trying to figure out if there is anything else that is needed (the environment is pretty complex).
We do have
tasks.withType(JavaCompile) {
options.fork = true
options.forkOptions.executable = 'javac' // we have this in one build tree but not another
options.encoding = 'UTF-8'
}
We got rid of all JDK9 arguments from the compilerArgs and jvmArgs - much too complicated.

Meanwhile, we would really like to see Gradle and Groovy fixed. I know that this approach to open packages is likely hiding a few more changes that are needed in the product code.

[blackdrag]

@sdfelts Goal of this is to be able to run a build with JDK9. Fixing general JDK9 problems of Groovy is not the goal here. For those it is probably better to run gradle with a JDK8 and run all the forked compilations and tests with a different JDK:

test {
   executable = "/path/to/jdk/bin/java"
}

and for compilation you already provided a way to use a different JDK. The Groovy problems with JDK9 I have seen so far (just normal classpath based projects without additional modules) seem not to have a problem with recent Groovy. I am very sure that is only because of the reluctant testing though. But for Groovy errors in Gradle...most of those are because here Groovy was used to bypass restrictions. There is no fix for these on the Groovy side short of rewriting java module descriptors on startup. And that is because of the way the module system is designed and its additional restrictions it imposes on everyone. And actually non of these problems are actually specific to Groovy. For most of these solutions you would probably have used reflection and it would have failed by now as well.

[sormuras]

@blackdrag Any new findings or even solutions regarding the "compiler daemon" issue? I'm stuck at a similar build-show-stopper here: https://travis-ci.org/junit-team/junit5/builds/219179434#L1029

Using "--no-daemon" on the initial "gradlew" call and:

	tasks.withType(JavaCompile) {
		options.fork = false
	}

	tasks.withType(GroovyCompile) {
		options.fork = false
	}

[blackdrag]

@sormuras try using a very recent jigsaw build and then

JAVA_OPTS='-XX:+IgnoreUnrecognizedVMOptions --permit-illegal-access'

plus in the gradle.properties:

org.gradle.jvmargs=-XX:+IgnoreUnrecognizedVMOptions --permit-illegal-access --show-version

I had not yet the time to actually try this out though. If you did try this, it would be very nice if you could report if it was of any help

[sormuras]

And leave the daemon "alive and kicking" enabled?

[sdfelts]

as I mentioned in my post above on March 7, I use _JAVA_OPTIONS to get JDK9 command line arguments from the command line to all embedded processes. We call gradle from the command line, ant, shell, etc. There are no problems running gradle or the related daemons (no need to turn them off using --no-daemon).

When b163 arrived, I put in a switch so that I can either run with the single --permit-illegal-access or the twenty five or so--add-opens. The --permit-illegal-access has the disadvantage that it prints out warnings even for places in the code that have fall-back code when the setAccessible fails. I've been building up a list of known cases that I can ignore. It's a great way to run the first time to get all of the problem packages in one pass instead of fixing one and re-running to the next one (annoying if you have hundreds of modules).

[sdfelts]

This is my gradle shell. It adds JDK9 options for running gradle to the other options needed for my project code. That allows me to isolate as much as possible the gradle/groovy dependencies.
We happen to have the Java version in an environment variable but you could always get it from java -version each time.
Note that the java.lang.ClassLoader.getPackages() problem mentioned in the link above is solved by using the --add-opens=java.base/java.lang=ALL-UNNAMED option or --permit-illegal-access.
The --add-opens that you need will depend on the classes and methods used in your gradle code. We have a lot of code and a lot of dependencies.

if [ "uname" = "Windows_NT" ]; then
GRADLE=gradle.bat
else
GRADLE=gradle
fi
case "$JAVAVER" in
1900*)
if [ x"$_JAVA_OPTIONS" = x--permit-illegal-access ]
then
_JAVA_OPTIONS=--permit-illegal-access
else
_JAVA_OPTIONS="$_JAVA_OPTIONS --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.text=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.regex=ALL-UNNAMED --add-opens=java.base/java.util.jar=ALL-UNNAMED --add-opens=java.base/java.util.zip=ALL-UNNAMED --add-opens=java.base/javax.net=ALL-UNNAMED --add-opens=java.base/sun.net.www=ALL-UNNAMED --add-opens=java.base/sun.net.www.protocol.file=ALL-UNNAMED --add-opens=java.base/sun.nio.fs=ALL-UNNAMED --add-opens=java.xml/javax.xml.transform=ALL-UNNAMED --add-opens=java.xml/javax.xml.transform.stream=ALL-UNNAMED --add-opens=java.xml/com.sun.org.apache.xalan.internal.xsltc=ALL-UNNAMED --add-opens=java.xml/com.sun.org.apache.xalan.internal.xsltc.compiler=ALL-UNNAMED --add-opens=java.xml/com.sun.org.apache.xalan.internal.xsltc.trax=ALL-UNNAMED --add-opens=java.base/java.security=ALL-UNNAMED"
fi
#_JAVA_OPTIONS="$_JAVA_OPTIONS -Dsun.reflect.debugModuleAccessChecks=true"
export _JAVA_OPTIONS
;;
esac
"$GRADLE_HOME/bin/$GRADLE" "$@"

[sormuras]

@sdfelts Thanks for the comments and hints. If I read your shell script and understand its usage correctly, it is used to initially start gradle with the vm options needed/wanted.

But what about the compile daemons @ghale talks about? What about vms that are spawend internally by Gradle? Subprojects, buildSrc folder? I can't get said vm options injected into those vms -- no matter if let the daemon live or not.

See, we have a "green" build with "--no-daemon": https://travis-ci.org/junit-team/junit5/builds/219396869
When I re-activate the junit-platform-gradle-plugin: https://travis-ci.org/junit-team/junit5/builds/219194962
The "opens" are no longer active, they're not passed down to the spawned "compileGroovy" task.

[blackdrag]

@sormuras setting the fork options for groovy compile does not work?

[blackdrag]

seems using _JAVA_OPTIONS works. Please note the leading underscore

[sdfelts]

I have been using _JAVA_OPTIONS to get gradle working for over a year.
Gradle seems to pass this environment to any daemons.
The only change that we made is to fork java (see the March 7 comment).

[sormuras]

@sormuras setting the fork options for groovy compile does not work?

It did not work. But after your " seems using _JAVA_OPTIONS works " I didn't investigate any further that path.

I have been using _JAVA_OPTIONS to get gradle working for over a year.

Thanks @sdfelts ! I totally forgot about that JDK built-in (undocumented?) feature -- even after you mentioned it here many times. Needed time to digest it. Now, the JUnit 5 build on Java 9 is "green" again: https://travis-ci.org/junit-team/junit5/builds/219592861

[sdfelts]

While _JAVA_OPTIONS is unsupported, there is now a supported equivalent.
It's not documented yet. The functionality is documented at http://ccc.us.oracle.com/8170832 but the official new name is JDK_JAVA_OPTIONS. It's available in b157. It prepends to the command line instead of appends and it supports @filename. I haven't tried it yet but I will be switching over soon (limiting us to run on b157 or later).

[sormuras]

Found the "external" link to the JDK_JAVA_OPTIONS issue: https://bugs.openjdk.java.net/browse/JDK-8173712
Incorporated it into our build already: https://travis-ci.org/junit-team/junit5/builds/219975618#L313

Thanks, again!

[oehme]

Is this still an issue with Gradle 4.1 and recent JDK 9 builds? Please try by removing all workarounds that you may have added previously.

[sdfelts]

The final JDK 9 build (b181) defaults to having all packages opened. All of these workarounds are no longer needed. However, a warning is still printed during the gradle code execution and it should be fixed.
The warning in worker.org.gradle.internal.reflect.JavaMethod
is tracked by #890

[oehme]

Thank you for verifying, let's continue on that other issue then.

[blackdrag]

while the jdk9 specific problem here got solved by jdk9 diverging from the originally planed logic - I still cannot set the jvm arguments of the gradle compiler daemon. I have not tested with Gradle 4.1 though.

[oehme]

The compiler daemons are configured via the options on the compile task. If that doesn't work, can you attach a small example that shows the problem?

[emartynov]

How to get memory dump with gradle daemon and javac? The next snippet doesn't work.

  tasks.withType(JavaCompile).all {
    options.forkOptions.jvmArgs += '-XX:+HeapDumpOnOutOfMemoryError'
  }

[csaltos]

In the case you want to go to the add-open JVM option, here's a command to find which module provide which package ->

java --list-modules | tr @ " " | awk '{ print $1 }' | xargs -n1 java -d

the name of the module will be shown with the @ while the name of the packages without it

(tested with JDK 11)