New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: run GC after Runner finished #145
Conversation
Thanks, could you verify that it actually solves #134 ? |
jmh.jar is still locking on windows. |
I tried File Leak Detector for both Gradle and JMH, but it didn't give me any results except non-closed socket in JMH. Which I ascribed to ShutdownTimeoutThread. @jnellis, waiting for 30 seconds, as I noted here, doesn't help too? |
Thanks for running that leak detector, the jar file was giving me manifest error. Trying to change This is the command line for the gradle daemon idle process:
These are the command lines for the two worker daemon processes that remain:
You can run the jmh task again. It's only when you try to Now, what if I kill these two worker processes but not the idle gradle daemon process and try to run At this point I decided to do a clean since I had killed those two processes and knew it would be able to delete that jmh.jar. NOTE: the original gradle daemon process is still sitting idle, I have not done
From this you can conclude that these two Gradle Worker Daemon processes (2 & 3) respectively map to the plugin tasks jmhRunByteCodeGenerator (me.champeau.gradle.JmhBytecodeGeneratorRunnable) and jmh (me.champeau.gradle.IsolatedRunner). So looking for commonalities of these two files I notice that both use javax.inject.@Inject along with JHMTask.java. Then I notice in JmhBytecodeGeneratorTask.java it uses import org.gradle.api.tasks.@CacheableTask. I don't know anything really about gradle plugin lifecycles and so haven't cloned the plugin to attempt building it but if you look at the javax.inject.@Inject javadocs it warns at the bottom about circular dependencies between fields. This seems a reasonable behavior between two related processes that can't shut down. Let's revert this perhaps and maybe revert As for my build file, it's basically straight out of the plugin example. |
@jnellis, thanks for inspecting processes. So, the problem is that this task is using Worker API, and Worker API does not support volatile classpath.
@jnellis, I didn't get your reasoning that the problem is in annotations. |
@grv87 Looks like that's it. It says Worker API was introduced in 4.1, that's where the locking behavior starts. For Windows you have to use gradle 4.0.2 and jmh plugin 0.4.3 right now for no issues. The WSL seems to work just fine though, you can see those same two Worker processes idle still, it just doesn't lock the jar file when trying to delete it during clean. Would these worker processes normally be exited if there wasn't this classpath issue? I suppose a workaround then for windows is that you can use a higher gradle version (with appropriate jmh plugin version) and then only call the jmh task from WSL. So far I haven't any conflicts going between the two except heavy memory use. |
There's no doubt to me that |
There's no amount of time that passes that it unlocks the jar on windows. It stays locked as long as those two worker daemon processes exist. Same on WSL(windows subsystem for linux; Ubuntu 16), except there is no lock to break(Linux don't care I guess), those worker process still sit idle waiting for IsolatedRunner and JmhBytecodeGeneratorRunnable to run again. There's |
@melix, I confirm that my previous statement on 30 seconds timeout was wrong. It was false positive, either because my system become limited on resources, and some idle processes decided to stop themselves, or this was a side effect from File Leak Detector. |
@melix What neccessitates using IsolationMode.PROCESS in these two tasks? Since killing these two processes seems to release the file lock in windows maybe not having them run in their own process in the first place is a solution. |
Using this isolation mode makes sure that the task workers work in isolation from Gradle itself (that is to say in a different process). So using a different isolation mode would only make things worse. To me the issue is JMH not closing a file. I didn't have time to investigate how to mitigate this, I will as soon as I can. |
@melix You should probably call Although there's no guaranty that this will fix the issue, it shouldn't make things worse. ;-) |
@veita This doesn't do anything from what I can tell. Locking still happens. I fork the plugin and publish the latest version to mavenLocal and then link to that in a project to test and this is what I've tried.
No. You can short circuit the IsolationRunner class to do nothing and the benchmark.jar file as well as all associated dependencies in the user home ./gradle/cache are locked. It does seem to be directly related to gradle worker api and how its used in this scenario; creating an intermediate dependency then using it. Which seems like it would be a common idiom. Furthermore, at least in WSL(ubuntu/oracle jdk8), zombie processes seem to be leftover. When a change in the source code or a project dependency invokes a new benchmark.jar being created; running that abandons the old daemon worker process that was still around and it spawns a new daemon worker process. So while its not blocking on WSL ubuntu it's also not allowing stale daemon processes to die off. The gradle docs say the default time deamon worker processes hang around until they are killed is three hours but that's not my experience; they will die only when @grv87 said something smart in the #937 issue. Why keep a hot path just to run benchmarks when the benchmark is required to be regenerated anytime the benched source code is changed. So while maybe this isn't a bug with the gradle worker api, it's probably the wrong use case to use considering how jmh has to rebuild the benchmark each time. To anyone that is having the same locking issues on windows and just has a simple project they would like to run jmh benchmarks until this is resolved I have made a crude drop in workaround |
This patch, used together with
forceGC=true
, fixes #134.