Gradle daemon causing build outcome problems and excessive memory usage

[mjcmatrix]

I have experienced, along with other developers, numerous problems with Grails apps built where the Gradle daemon has been involved.

I have so far hesitated to raise an issue for this because I have been unable to isolate a specific reproduceable case.

However, these problems seem to be have been persistent over the course of using all Grails 3.0 - 3.1.1 versions. (Graeme has asked me to file an issue.)

I have the Gradle daemon disabled outside of Grails, due to non-specific issues like these (and excessive memory use resulting from multiple background Java processes). However, the grails command still starts up Gradle daemons.

It looks like there is a way to make Gradle embedded, instead of using the daemon. This could be a solution: http://gradle.1045684.n5.nabble.com/Tooling-API-stop-daemon-td4375065.html

Issues reported with the daemon running include:

• Unexpected or unfathomable build outcomes. For example,
  rlovtangen reported on Slack #questions: a simple grails create app -> grails war resulted in an NPE exception in assetCompile (on Grails 3.1.1) and failed build.
  • ./gradlew --stop && ./gradlew clean && ./gradlew assemble made this problem go away.
• Improper grails clean - where a code file can be deleted but appears in the next build assembly anyway after clean.
  • rm -rf build/ has to be ran on occasion and this should not be necessary.
• If the main Application class is moved to a different package, despite a full clean and rebuild cycle bootRun can fail to find the main class for several subsequent builds and can prevent application startup.
  • This eventually stops, probably after a workaround sequence such as ./gradlew --stop && ./gradlew clean && ./gradlew build has been applied.
• Moving between building various different test apps (main app and small test apps) when trying to debug or to isolate a weird startup fault, seems to cross-contaminate in some unexplainable circumstances. Suddently the same unexpected startup fault that is occurring in an main app can start occurring in a simple test app being used to isolate the fault, an app that previously worked. The fault can then can disappear again several builds later without code changes. This has happened to me several times.
• WAR builds [for me, targeting Tomcat 7] that periodically do not start up properly (with no log output whatsoever), then a couple of builds later (with no code changes), work fine again.
• The application can be bought to a stable state, committed to git, then go home, git pull and try to build and run the same war on a laptop in a near-identical environment and it won't start up properly.

A second reason to integrate ./gradlew --stop into the start of all my convenience build shell scripts and stop using the grails command in all cases has been the more general issue with Gradle daemons:

• At least one Gradle daemon seems to be started for each project. When jumping around several test projects and main projects or within a Gradle multi-build, this can end up resulting in several background Java processes running and cause slowness / swapping / excess memory usage issues because there are 4-5 unnecessary Java processes accumulating (in addition to the IDE and run-app or container JVM). (This is the main reason I have Gradle daemon disabled.) (My Ubuntu dev boxes have 8GB RAM. While developing in Grails 2.4 or without the daemon, this is sufficient RAM.)

Typically, when these weird scenarios are encountered, ./gradlew --stop && rm -rf build/ && ./gradlew clean, then using only ./gradlew commands to build projects from scratch (never grails commands), can make a build and/or deployed application startup work as expected.

Given that Grails 3.1 currently uses Gradle daemon (even if the user has it turned off in their system gradle.properties), and when I see several people having issues like I am, I think we should be concerned. These are frustrating issues when they are encountered and many hours are lost. This seems to impact both new users of Grails 3 and Grails developers who are trying to migrate 2.x apps to 3.x. The latter group are already debugging a long series of application start-up problems and making build script changes, configuration changes and re-factoring code as they go along, trying to migrate their apps to get them to eventually successfully start without any remaining issues.

[graemerocher]

This is now fixed when running the grails command, however when using the gradle command you will have to disable the daemon via configuration in gradle.properties. See https://docs.gradle.org/current/userguide/gradle_daemon.html

[graemerocher]

There are issues with output in older versions of Gradle so this is only going into the 3.1.x branch and not 3.0.x