Skip to content

Loading…

fix the thread leak in timeout class (https://github.com/cucumber/cucumb... #640

Merged
merged 2 commits into from

5 participants

@volna80

fix the thread leak in timeout class (cucumber#639)

1) all tests are passed

2) tested with my own business project which has a lot of scenarios, used yourkit to profile and confirmed that no thread leak now.

@ffbit ffbit commented on the diff
core/src/main/java/cucumber/runtime/Timeout.java
@@ -28,6 +27,7 @@ public void run() {
} finally {
done.set(true);
timer.cancel(true);
+ executorService.shutdownNow();
@ffbit Cucumber member
ffbit added a note

This line of code only makes the real thing.

@volna80
volna80 added a note

sure,

PS. haven't worked with github pull request. How can I do it? cancel pull request or raise new one?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@ffbit ffbit commented on an outdated diff
core/src/main/java/cucumber/runtime/Timeout.java
@@ -1,9 +1,6 @@
package cucumber.runtime;
-import java.util.concurrent.Executors;
-import java.util.concurrent.ScheduledFuture;
-import java.util.concurrent.TimeUnit;
-import java.util.concurrent.TimeoutException;
+import java.util.concurrent.*;
@ffbit Cucumber member
ffbit added a note

Could you please consider reverting of these changes - wild card import isn't a project's common practice.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@ffbit
Cucumber member

There at least 2 ways (in my humble opinion):

  • ammend an existent pulled commit in PR
  • make another commit in the same branch (you've already done this)

Also spend five minutes on reading the contribution guide.

Thanks for your contribution.

@volna80

Hi,

Are you expecting anything else for this PR?

I see that the build has finished with Error state, but couldn't understand what the error. I've seen that the build has passed according to maven output. Could somebody look at this? Is it possible to restart a build?

@brasmusson

@volna80 For us with write access to the cucumber-jvm repository it is possible to restart a build on Travis. I have restarted the job in the error state.

An advanced tip for pull requests on github:
If you edit the pull request body and include fixes #639 (or one of the other "closing" keywords), then issue #639 will automatically be closed when this pull request is merged, see https://help.github.com/articles/closing-issues-via-commit-messages#closing-issues-with-pull-requests

@aslakhellesoy
Cucumber member

Are you expecting anything else for this PR?

Yes - a unit test. You can use Thread.getAllStackTraces() to ensure that the number of threads remains constant after say, 100 invocations of timeout(). Also verify that no exceptions are thrown. @os97673 mentioned a risk of InterruptedException being thrown unless done.set(true) is called inside the try block.

@aslakhellesoy
Cucumber member

Sorry - I spoke too soon! I see there is a unit test now :-) I'll leave some comments in that test instead.

@aslakhellesoy aslakhellesoy commented on the diff
core/src/test/java/cucumber/runtime/TimeoutTest.java
((7 lines not shown))
+ final long startNumberOfThreads = Thread.getAllStackTraces().size();
+
+ for(int i = 0; i < 1000; i++){
+ Timeout.timeout(new Timeout.Callback<String>() {
+ @Override
+ public String call() throws Throwable {
+ return null;
+ }
+ }, 10);
+ }
+ Thread.sleep(10);
+
+ final long finishNumberOfThreads = Thread.getAllStackTraces().size();
+
+ assertTrue("The number of threads have grown significantly. start: " + startNumberOfThreads + ", end:" + finishNumberOfThreads,
+ Math.abs(finishNumberOfThreads - startNumberOfThreads) < 5);
@aslakhellesoy Cucumber member

This is a fragile test that will eventually fail on a particularly rainy day. It's what we call a flickering test.

I'm guessing you picked the number 5 arbitrarily because you saw the number of threads increase. Why has the number of threads increased by 5? Maybe because Timeout creates a new executor every time timeout() is called. Try making the executor a private static field so there is only ever one, then see if you can get the thread difference down to 1.

To me an assertEquals(startNumberOfThreads, finishNumberOfThreads) seems to work, when the test case is given a bit extra time than the initial 10ms (see comment above).

@os97673 Cucumber member
os97673 added a note

imho to make the code unit-testable you have to inject the executor (or something else) otherwise the test will be either unstable or slow.

@volna80
volna80 added a note

Frankly speaking,

I don't think that unit tests are design to catch such issues (like any resource leaks). Than we talk about real multi threading, obviously there is no guarantee from jvm that it will stop all threads in a given timeout. We don't know how exactly will behave Thread.getAllStackTraces() in future java versions and etc.

You want to have a unit test, I"ve written one. I believe it will be a stable. If you ask me again, I prefer just to delete this test and fix the bug.

The goal of this test that just to check that it doesn't create a huge amount of new threads.

@aslakhellesoy Cucumber member

Good points @volna80. Everything is merged and on master now. If this starts failing at some point I agree we can delete the test. For now, let's just leave it there - it seems ok.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@brasmusson brasmusson commented on the diff
core/src/test/java/cucumber/runtime/TimeoutTest.java
((4 lines not shown))
+ @Test
+ public void no_thread_leak() throws Throwable{
+
+ final long startNumberOfThreads = Thread.getAllStackTraces().size();
+
+ for(int i = 0; i < 1000; i++){
+ Timeout.timeout(new Timeout.Callback<String>() {
+ @Override
+ public String call() throws Throwable {
+ return null;
+ }
+ }, 10);
+ }
+ Thread.sleep(10);
+
+ final long finishNumberOfThreads = Thread.getAllStackTraces().size();

It is a bit optimistic to think that all timeout threads have been deleted directly after 10ms (at least on Windows where I experimented to figure out if the < 5 was necessary). A safer construct is something like:

        long finishNumberOfThreads = Thread.getAllStackTraces().size();
        for (int i = 0; i < 100; i++) {
            if (finishNumberOfThreads == startNumberOfThreads) {
                break;
            }
            Thread.sleep(5);
            finishNumberOfThreads = Thread.getAllStackTraces().size();
        }

It allows for up to 500ms before the number of threads are back to the starting level, but does not sleep longer then on average 2,5ms longer than necessary. So far I have never seen more than a couple of loops actually been needed in practice.

@volna80
volna80 added a note

I won't be so optimistics and predict that any other thread won't spawn one more new thread. We don't control the env. I think we should just focus on the main thing. It doesn't spawn enormous number of new threads (thread leak detect).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@aslakhellesoy aslakhellesoy merged commit 5ca341d into cucumber:master

1 check passed

Details default The Travis CI build passed
@aslakhellesoy
Cucumber member

I merged this in with some changes. If anyone from the team wants to improve it, please feel free. It runs pretty fast though, and I think it's pretty reliable now, thanks to @brasmusson's suggestion.

@brasmusson

@aslakhellesoy The TimeoutTest has been failing from time to time, for me more often when executed from Eclipse than when executed from maven. Now it also has failed on Travis (https://travis-ci.org/cucumber/cucumber-jvm/builds/15295806). One source of failure I tracked down was that some (one) extra pool threads exist when the initial thread count is samples, so the thread count is actually decreasing (slightly) during the test. I changed the test to allow for decreasing number of threads in 77a93d1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
This page is out of date. Refresh to see the latest.
View
6 core/src/main/java/cucumber/runtime/Timeout.java
@@ -2,6 +2,7 @@
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import java.util.concurrent.atomic.AtomicBoolean;
@@ -13,7 +14,9 @@
} else {
final Thread executionThread = Thread.currentThread();
final AtomicBoolean done = new AtomicBoolean();
- ScheduledFuture<?> timer = Executors.newSingleThreadScheduledExecutor().schedule(new Runnable() {
+
+ ScheduledExecutorService executorService = Executors.newSingleThreadScheduledExecutor();
+ ScheduledFuture<?> timer = executorService.schedule(new Runnable() {
@Override
public void run() {
if (!done.get()) {
@@ -28,6 +31,7 @@ public void run() {
} finally {
done.set(true);
timer.cancel(true);
+ executorService.shutdownNow();
@ffbit Cucumber member
ffbit added a note

This line of code only makes the real thing.

@volna80
volna80 added a note

sure,

PS. haven't worked with github pull request. How can I do it? cancel pull request or raise new one?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
}
}
View
26 core/src/test/java/cucumber/runtime/TimeoutTest.java
@@ -6,6 +6,7 @@
import static java.lang.Thread.sleep;
import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertTrue;
import static org.junit.Assert.fail;
public class TimeoutTest {
@@ -33,6 +34,28 @@ public String call() throws Throwable {
fail();
}
+ @Test
+ public void no_thread_leak() throws Throwable{
+
+ final long startNumberOfThreads = Thread.getAllStackTraces().size();
+
+ for(int i = 0; i < 1000; i++){
+ Timeout.timeout(new Timeout.Callback<String>() {
+ @Override
+ public String call() throws Throwable {
+ return null;
+ }
+ }, 10);
+ }
+ Thread.sleep(10);
+
+ final long finishNumberOfThreads = Thread.getAllStackTraces().size();

It is a bit optimistic to think that all timeout threads have been deleted directly after 10ms (at least on Windows where I experimented to figure out if the < 5 was necessary). A safer construct is something like:

        long finishNumberOfThreads = Thread.getAllStackTraces().size();
        for (int i = 0; i < 100; i++) {
            if (finishNumberOfThreads == startNumberOfThreads) {
                break;
            }
            Thread.sleep(5);
            finishNumberOfThreads = Thread.getAllStackTraces().size();
        }

It allows for up to 500ms before the number of threads are back to the starting level, but does not sleep longer then on average 2,5ms longer than necessary. So far I have never seen more than a couple of loops actually been needed in practice.

@volna80
volna80 added a note

I won't be so optimistics and predict that any other thread won't spawn one more new thread. We don't control the env. I think we should just focus on the main thing. It doesn't spawn enormous number of new threads (thread leak detect).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+
+ assertTrue("The number of threads have grown significantly. start: " + startNumberOfThreads + ", end:" + finishNumberOfThreads,
+ Math.abs(finishNumberOfThreads - startNumberOfThreads) < 5);
@aslakhellesoy Cucumber member

This is a fragile test that will eventually fail on a particularly rainy day. It's what we call a flickering test.

I'm guessing you picked the number 5 arbitrarily because you saw the number of threads increase. Why has the number of threads increased by 5? Maybe because Timeout creates a new executor every time timeout() is called. Try making the executor a private static field so there is only ever one, then see if you can get the thread difference down to 1.

To me an assertEquals(startNumberOfThreads, finishNumberOfThreads) seems to work, when the test case is given a bit extra time than the initial 10ms (see comment above).

@os97673 Cucumber member
os97673 added a note

imho to make the code unit-testable you have to inject the executor (or something else) otherwise the test will be either unstable or slow.

@volna80
volna80 added a note

Frankly speaking,

I don't think that unit tests are design to catch such issues (like any resource leaks). Than we talk about real multi threading, obviously there is no guarantee from jvm that it will stop all threads in a given timeout. We don't know how exactly will behave Thread.getAllStackTraces() in future java versions and etc.

You want to have a unit test, I"ve written one. I believe it will be a stable. If you ask me again, I prefer just to delete this test and fix the bug.

The goal of this test that just to check that it doesn't create a huge amount of new threads.

@aslakhellesoy Cucumber member

Good points @volna80. Everything is merged and on master now. If this starts failing at some point I agree we can delete the test. For now, let's just leave it there - it seems ok.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+
+ }
+
@Test(expected = TimeoutException.class)
public void times_out_infinite_loop_if_it_takes_too_long() throws Throwable {
final Slow slow = new Slow();
@@ -46,6 +69,9 @@ public Void call() throws Throwable {
fail();
}
+
+
+
public static class Slow {
public String slow() throws InterruptedException {
sleep(10);
Something went wrong with that request. Please try again.