Long-running tests ignored

[AndrewInScotland]

We have some integration tests that can take longer than 2 minutes on our TFS build machine. These are not reported by the test runner.
I have confirmed that this also occurs in VS 2012 Update 2.
It looks like any test that takes longer than 180 seconds is not reported!
Try the below test fixture. Tests "Quick" and "Slow" will be executed and reported; test "Slower" does not appear to even execute, and is shown as 'dimmed' in the Test Explorer window.
Note that all 3 tests will run and get correctly reported by NUnit Console.

[TestFixture]
public class TimeoutTest
{
    [Test]
    public void Quick()
    {
        Thread.Sleep(1);
    }

    [Test]
    public void Slow()
    {
        Thread.Sleep(150000);
    }

    [Test]
    public void Slower()
    {
        Thread.Sleep(250000);
    }
}

[GoppeltM]

I can verify that for VS 2012 Update 4 as well. However, I had to increase the time to 10 mins, 4 were not enough (CPU cycle limited?). Thanks for the info, I already spent multiple days with this issue with no clue where to look. At least now it's obvious that this is a test runner issue.

[OsirisTerje]

Hi !
I have checked this and can confirm it.
I checked this both locally on a TFS 2013 server onpremises, The same tests on MSTest (running with VSTEst runner) works as they should, the NUnit not. So it is either an adapter bug or a NUnit runner bug. There are no messages in the test output.

[CharliePoole]

Hi Terje,

Given that it doesn't happen under the NUnit console or gui runners, I'd
say it's unlikely to be NUnit. Our adapter doesn't do anything but wait for
the test to complete, so all that seems to be left is that there may be
some VS setting we are not making use of, which MSTest uses.

On Thu, Feb 13, 2014 at 2:38 PM, Terje Sandstrom
notifications@github.comwrote:

Hi !
I have checked this and can confirm it.
I checked this both locally on a TFS 2013 server onpremises, The same
tests on MSTest (running with VSTEst runner) works as they should, the
NUnit not. So it is either an adapter bug or a NUnit runner bug.

Reply to this email directly or view it on GitHubhttps://github.com//issues/24#issuecomment-35035114
.

[ggohierroy]

As part of our TFS build, we added a post-build event to run the long-running tests in the nunit command line tool and dump them into a text file. There doesn't seem to be another alternative except switching to MSTest. Is there any update or clues of what the cause might be since the bug was reported?

[sandshadow]

The root issue is that NUnitEventListener is a MarshalByRefObject. This means it has a default lifetime lease of 5 minutes, with an extension of 2 minutes every time it receives a callback.
http://msdn.microsoft.com/en-us/magazine/cc300474.aspx#edupdate
If you look at the listener's non-public members in the debugger, you will see it has an expired lease at the time the call to runner.run completes early.
I've tested switching to an infinite lifetime lease for the listener and it fixes the bug, but I need to verify it does so without introducing leaks.

[ggohierroy]

Awesome news!

[sandshadow]

I have a pull request out for the fix.

[OsirisTerje]

I have merged in the PR now, and I will add this to the 1.2 release. I will also build a bugfix-release tomorrow with this fix included, for testing/verification.

[sandshadow]

Hello Terje,

Do you have a bugfix release package available for download?

Thanks,

Eric

From: Terje Sandstrom [mailto:notifications@github.com]
Sent: Monday, July 7, 2014 2:43 PM
To: nunit/nunit-vs-adapter
Cc: Eric Dettinger
Subject: Re: [nunit-vs-adapter] Long-running tests ignored (#24)

I have merged in the PR now, and I will add this to the 1.2 release. I will also build a bugfix-release tomorrow with this fix included, for testing/verification.

—
Reply to this email directly or view it on GitHub #24 (comment) . https://github.com/notifications/beacon/7919106__eyJzY29wZSI6Ik5ld3NpZXM6QmVhY29uIiwiZXhwaXJlcyI6MTcyMDM4ODYwMiwiZGF0YSI6eyJpZCI6MjQ4MzUyNDh9fQ==--523d07f197987bdbdc451a6ccb7121573b5b9465.gif

[OsirisTerje]

Hi!
Uploaded a bugfix release here: https://bintray.com/nunit/NUnitAdapter/NUnitAdapter/1.1.0.3/view/files

(Sorry for the delay - vacation issues :-) )