Test with infinite loop in TearDown cannot be aborted

[CharliePoole]

Pursuant to a discussion on issue #326 I added some tests in order to see the behavior of tests that caused a timeout because of a hung SetUp or a hung TearDown. An infinite loop in the SetUp is handled the same as one in the test but a loop in TearDown causes the entire run to hang.

The cause of this phenomenon is that we run TearDown out of a finally clause. Code in a finally clause is not interupted by a ThreadAbortException.

The same behavior occurs in NUnit V2. See nunit/nunitv2#26

It appears that there are three options:

1. Don't run TearDown in a finally clause.
2. Don't use Thread.Abort to cancel a test that times out.
3. Have NUnit host the CLR and thereby gain the ability to escalate aborts.

All of these are fairly big deals, so some design work is needed.

[oznetmaster]

There is my option as well. Have separate timeouts for the test and for the entire test process.

[CharliePoole]

It seems to me that having two different timeouts would just give us one
that worked and one that didn't, at least until we solve the problem of
interupting teardown.

On Mon, Nov 10, 2014 at 1:34 PM, oznetmaster notifications@github.com
wrote:

There is my option as well. Have separate timeouts for the test and for
the entire test process.

From: CharliePoole [mailto:notifications@github.com]
Sent: Tuesday, November 11, 2014 3:59 AM
To: nunit/nunit
Subject: [nunit] Test with infinite loop in TearDown cannot be aborted
(#352)

Pursuant to a discussion on issue #326 <
https://github.com/nunit/nunit/issues/326> I added some tests in order to
see the behavior of tests that caused a timeout because of a hung SetUp or
a hung TearDown. An infinite loop in the SetUp is handled the same as one
in the test but a loop in TearDown causes the entire run to hang.

The cause of this phenomenon is that we run TearDown out of a finally
clause. Code in a finally clause is not interupted by a
ThreadAbortException.

The same behavior occurs in NUnit V2. See nunt/nunitv2/#26 <
https://github.com/nunit/nunit/issues/26> .

It appears that there are three options:

1. Don't run TearDown in a finally clause.
2. Don't use Thread.Abort to cancel a test that times out.
3. Have NUnit host the CLR and thereby gain the ability to escalate
   aborts.

All of these are fairly big deals, so some design work is needed.

—
Reply to this email directly or view it on GitHub <
https://github.com/nunit/nunit/issues/352> . <
https://github.com/notifications/beacon/AFh64SRakRCigN2YOXke_9XM5OtId2v7ks5nMRBqgaJpZM4C5QQI.gif>

—
Reply to this email directly or view it on GitHub
#352 (comment).

[oznetmaster]

Not sure why that would be.

If the test timeout specifically encompassed only the actual test, and not the setup or teardown, it would not have a finally clause problem.

If the test process timeout wrapped the entire process, then it would not have a finally clause problem.

The reason is that there would no longer be a catch (ThreadAbortException) clause required. The finally clause is there only to handle the ThreadAbortException.

[CharliePoole]

That's not the finally clause causing the problem... we can easily get rid of it if necessary.

Teardown is kicked off in SetUpTearDownCommand.cs at line 79.

All versions of NUnit since way back have used a finally clause to run teardown.

So, what I'm saying is that your proposed change is about what to do after we gain control and it's worth discussing. But this issue is about the fact that we don't gain control in a specific situation.

But looking at and talking about code has its limits. If you add the tests to your build, you can prove or disprove what's happening.

[oznetmaster]

I added your tests to my build. The TearDownTimesOutAndNoFurtherTearDownIsRun which you had to inhibit works perfectly on my implementation. So it appears that additional code I added previously solves the problem of teardown timeouts.

SetUpTimesOutAndTearDownIsRun passes with no issues.

[CharliePoole]

Huh! Can you figure out the minimal change to make it work? I'd prefer not
to have to add all those fields to TestExecutionContext and give WorkItem
so much visibility into the internals of a test.

On Tue, Nov 11, 2014 at 12:15 PM, oznetmaster notifications@github.com
wrote:

I added your tests to my build. The
TearDownTimesOutAndNoFurtherTearDownIsRun which you had to inhibit works
perfectly on my implementation. So it appears that additional code I added
previously solves the problem of teardown timeouts.

SetUpTimesOutAndTearDownIsRun passes with no issues.

From: CharliePoole [mailto:notifications@github.com]
Sent: Tuesday, November 11, 2014 8:18 AM
To: nunit/nunit
Cc: oznetmaster
Subject: Re: [nunit] Test with infinite loop in TearDown cannot be aborted
(#352)

That's not the finally clause causing the problem... we can easily get rid
of it if necessary.

Teardown is kicked off in SetUpTearDownCommand.cs at line 79.

All versions of NUnit since way back have used a finally clause to run
teardown.

So, what I'm saying is that your proposed change is about what to do after
we gain control and it's worth discussing. But this issue is about the fact
that we don't gain control in a specific situation.

But looking at and talking about code has its limits. If you add the tests
to your build, you can prove or disprove what's happening.

—
Reply to this email directly or view it on GitHub <
https://github.com/nunit/nunit/issues/352#issuecomment-62481091> . <
https://github.com/notifications/beacon/AFh64WLp7haQELi0ZBXRHL4Hj0Cg69kIks5nMU1AgaJpZM4C5QQI.gif>

—
Reply to this email directly or view it on GitHub
#352 (comment).

[oznetmaster]

I will look into it further.

[oznetmaster]

Well, this is a real weird one, and it had nothing to do with NUnit! There is a bug in the Crestron CF wrapper that causes Thread.Join() to return immediately after a Thread.Abort has been executed, rather than waiting until the abort has actually completed. When I added a wait for the abort to actually complete, all the problems and issues went away, and I now see identical behavior to the normal CF build. This includes the TearDown infinite loop behavior. Without this fix, the teardown infinite loop just kept running and the rest of the tests kept right one going, hence the apparent success.

Sorry about this. I have reported this bug to Crestron as well, but a simple wait loop in WorkItem has fixed it for me for now.