Runner occasionally showing error messages about trying to access unloaded app domains #490

Closed
bradwilson opened this Issue Jul 27, 2015 · 15 comments

Comments

Projects
None yet
8 participants
@bradwilson
Member

bradwilson commented Jul 27, 2015

From @bradwilson on June 30, 2014 6:18

(Moved from #33)

Copied from original issue: xunit/visualstudio.xunit#6

@bradwilson

This comment has been minimized.

Show comment
Hide comment
@bradwilson

bradwilson Jul 27, 2015

Member

From @charleskorn on August 10, 2014 3:2

I can verify that I am occasionally seeing this issue as well.

Member

bradwilson commented Jul 27, 2015

From @charleskorn on August 10, 2014 3:2

I can verify that I am occasionally seeing this issue as well.

@bradwilson

This comment has been minimized.

Show comment
Hide comment
@bradwilson

bradwilson Jul 27, 2015

Member

From @mring82 on December 8, 2014 16:27

I'm able to repro this issue on my machine consistently using the Visual Studio Test Runner and the latest pre-release bits.

  • VS 2013 Premium Update 4
  • .NET 4.5 Class Library Project

Have installed the following NuGet packages into the test project (per the Getting Started guide):

<?xml version="1.0" encoding="utf-8"?>
<packages>
  <package id="xunit" version="2.0.0-beta-build2700" targetFramework="net45" />
  <package id="xunit.abstractions" version="2.0.0-beta-build2700" targetFramework="net45" />
  <package id="xunit.assert" version="2.0.0-beta-build2700" targetFramework="net45" />
  <package id="xunit.core" version="2.0.0-beta-build2700" targetFramework="net45" />
  <package id="xunit.runner.visualstudio" version="0.99.9-build1021" targetFramework="net45" />
</packages>

I was getting this when running a bunch of Theory test methods, so I removed all of that and tried running a simple Fact method:

[Fact]
public void RunMyTest()
{
  Assert.True(1 == 1);
}

No change. I still get the error. Here's what is from my Output window:

------ Run test started ------
[xUnit.net 00:00:00.0000057] Execution started
[xUnit.net 00:00:00.0026968] Settings: MaxParallelThreads = 0, NameDisplay = Short, ParallelizeAssemblies = False, ParallelizeTestCollections = True, ShutdownAfterRun = False
[xUnit.net 00:00:00.0037285] Discovery started
[xUnit.net 00:00:00.3900033] Discovery starting: MyTestAssembly.Tests.dll
[xUnit.net 00:00:00.5129828] Discovery finished: MyTestAssembly.Tests.dll (1 tests)
[xUnit.net 00:00:00.5613374] Discovery complete
[xUnit.net 00:00:00.5613374] Execution starting: MyTestAssembly.Tests.dll
[xUnit.net 00:00:00.5613374] Execution finished: MyTestAssembly.Tests.dll
System.AppDomainUnloadedException: Attempted to access an unloaded AppDomain. This can happen if the test(s) started a thread but did not stop it. Make sure that all the threads started by the test(s) are stopped before completion.
System.AppDomainUnloadedException: Attempted to access an unloaded AppDomain. This can happen if the test(s) started a thread but did not stop it. Make sure that all the threads started by the test(s) are stopped before completion.
[xUnit.net 00:00:00.5613374] Execution complete
No test is available in C:\Git\MyTestAssembly.Tests\bin\Debug\MyTestAssembly.Tests.dll. Make sure that installed test discoverers & executors, platform & framework version settings are appropriate and try again.
========== Run test finished: 0 run (0:00:01.37) ==========

Thoughts?

Member

bradwilson commented Jul 27, 2015

From @mring82 on December 8, 2014 16:27

I'm able to repro this issue on my machine consistently using the Visual Studio Test Runner and the latest pre-release bits.

  • VS 2013 Premium Update 4
  • .NET 4.5 Class Library Project

Have installed the following NuGet packages into the test project (per the Getting Started guide):

<?xml version="1.0" encoding="utf-8"?>
<packages>
  <package id="xunit" version="2.0.0-beta-build2700" targetFramework="net45" />
  <package id="xunit.abstractions" version="2.0.0-beta-build2700" targetFramework="net45" />
  <package id="xunit.assert" version="2.0.0-beta-build2700" targetFramework="net45" />
  <package id="xunit.core" version="2.0.0-beta-build2700" targetFramework="net45" />
  <package id="xunit.runner.visualstudio" version="0.99.9-build1021" targetFramework="net45" />
</packages>

I was getting this when running a bunch of Theory test methods, so I removed all of that and tried running a simple Fact method:

[Fact]
public void RunMyTest()
{
  Assert.True(1 == 1);
}

No change. I still get the error. Here's what is from my Output window:

------ Run test started ------
[xUnit.net 00:00:00.0000057] Execution started
[xUnit.net 00:00:00.0026968] Settings: MaxParallelThreads = 0, NameDisplay = Short, ParallelizeAssemblies = False, ParallelizeTestCollections = True, ShutdownAfterRun = False
[xUnit.net 00:00:00.0037285] Discovery started
[xUnit.net 00:00:00.3900033] Discovery starting: MyTestAssembly.Tests.dll
[xUnit.net 00:00:00.5129828] Discovery finished: MyTestAssembly.Tests.dll (1 tests)
[xUnit.net 00:00:00.5613374] Discovery complete
[xUnit.net 00:00:00.5613374] Execution starting: MyTestAssembly.Tests.dll
[xUnit.net 00:00:00.5613374] Execution finished: MyTestAssembly.Tests.dll
System.AppDomainUnloadedException: Attempted to access an unloaded AppDomain. This can happen if the test(s) started a thread but did not stop it. Make sure that all the threads started by the test(s) are stopped before completion.
System.AppDomainUnloadedException: Attempted to access an unloaded AppDomain. This can happen if the test(s) started a thread but did not stop it. Make sure that all the threads started by the test(s) are stopped before completion.
[xUnit.net 00:00:00.5613374] Execution complete
No test is available in C:\Git\MyTestAssembly.Tests\bin\Debug\MyTestAssembly.Tests.dll. Make sure that installed test discoverers & executors, platform & framework version settings are appropriate and try again.
========== Run test finished: 0 run (0:00:01.37) ==========

Thoughts?

@bradwilson

This comment has been minimized.

Show comment
Hide comment
@bradwilson

bradwilson Jul 27, 2015

Member

We're not sure what the issue is yet. It does not appear to have any impact other than this display (all tests run and report correctly), so it's been a fairly low priority issue for us.

Also, it's worth noting that build 2700 is beta 3. We have released 2 newer betas since then. :)

Member

bradwilson commented Jul 27, 2015

We're not sure what the issue is yet. It does not appear to have any impact other than this display (all tests run and report correctly), so it's been a fairly low priority issue for us.

Also, it's worth noting that build 2700 is beta 3. We have released 2 newer betas since then. :)

@bradwilson

This comment has been minimized.

Show comment
Hide comment
@bradwilson

bradwilson Jul 27, 2015

Member

From @mring82 on December 8, 2014 17:5

Interesting... When I ran this NuGet command again...

Install-Package xunit -Pre

... it was still pulling down build 2700. But when I explicitly specified the version...

Install-Package xunit -Version 2.0.0-beta5-build2785 -Pre

... it pulled down the latest. Strange...

Ok, after pulling down the latest package versions, I am now able to run my tests within Test Explorer and get back results.

Thanks!
Matt

Member

bradwilson commented Jul 27, 2015

From @mring82 on December 8, 2014 17:5

Interesting... When I ran this NuGet command again...

Install-Package xunit -Pre

... it was still pulling down build 2700. But when I explicitly specified the version...

Install-Package xunit -Version 2.0.0-beta5-build2785 -Pre

... it pulled down the latest. Strange...

Ok, after pulling down the latest package versions, I am now able to run my tests within Test Explorer and get back results.

Thanks!
Matt

@bradwilson

This comment has been minimized.

Show comment
Hide comment
@bradwilson

bradwilson Jul 27, 2015

Member

From @giggio on January 23, 2015 12:3

Just FYI: I just got that error once on VS 2015 CTP 5, with runner 2.0.0-rc1-build1030. It ran the tests, but showed the message on the Output window. It is not happening anymore.

Member

bradwilson commented Jul 27, 2015

From @giggio on January 23, 2015 12:3

Just FYI: I just got that error once on VS 2015 CTP 5, with runner 2.0.0-rc1-build1030. It ran the tests, but showed the message on the Output window. It is not happening anymore.

@bradwilson

This comment has been minimized.

Show comment
Hide comment
@bradwilson

bradwilson Jul 27, 2015

Member

From @khayes on March 14, 2015 15:10

I'm also experiencing this when running tests with xunit.console.clr4.x86.exe. In my particular case I'm invoking the console runner via powershell.

The dll I am testing is as simple as it can get, it only contains one test which is the following:

[Fact]
[Trait("Category", "UnitTest")]
public void TestMethod()
{
Assert.True(true);
}

Member

bradwilson commented Jul 27, 2015

From @khayes on March 14, 2015 15:10

I'm also experiencing this when running tests with xunit.console.clr4.x86.exe. In my particular case I'm invoking the console runner via powershell.

The dll I am testing is as simple as it can get, it only contains one test which is the following:

[Fact]
[Trait("Category", "UnitTest")]
public void TestMethod()
{
Assert.True(true);
}

@bradwilson

This comment has been minimized.

Show comment
Hide comment
@bradwilson

bradwilson Jul 27, 2015

Member

From @DerAlbertCom on May 20, 2015 20:1

Happens also in Visual Studio Online (current non pre bits)

Member

bradwilson commented Jul 27, 2015

From @DerAlbertCom on May 20, 2015 20:1

Happens also in Visual Studio Online (current non pre bits)

@giggio

This comment has been minimized.

Show comment
Hide comment
@giggio

giggio Jul 27, 2015

@bradwilson It seems that you are now concentrating all issues in this repo, right?

giggio commented Jul 27, 2015

@bradwilson It seems that you are now concentrating all issues in this repo, right?

@bradwilson

This comment has been minimized.

Show comment
Hide comment
@bradwilson

bradwilson Jul 27, 2015

Member

That is correct.

https://twitter.com/xunit/status/625773679389970432

I have closed issues in all the other repos that I own, and added a comment to the repo description saying where to log issues.

Member

bradwilson commented Jul 27, 2015

That is correct.

https://twitter.com/xunit/status/625773679389970432

I have closed issues in all the other repos that I own, and added a comment to the repo description saying where to log issues.

@joelwilson

This comment has been minimized.

Show comment
Hide comment
@joelwilson

joelwilson Aug 28, 2015

I don't know if it's been mentioned elsewhere, but I ran into this error when I had some accidental recursion going on.

I don't know if it's been mentioned elsewhere, but I ran into this error when I had some accidental recursion going on.

@Talento90

This comment has been minimized.

Show comment
Hide comment
@Talento90

Talento90 Sep 28, 2015

This problem still occurs in version 2.1 :(

This problem still occurs in version 2.1 :(

@thebothead

This comment has been minimized.

Show comment
Hide comment
@thebothead

thebothead Oct 27, 2015

This seems to happen at random intermittent points in the test discovery and executing stages. This snippet from the log is happening well before the log indicates it is starting any tests. This occurs in both VSO and locally in Visual Studio. I do suspect that there is a leaked task in there somewhere in my tests, but the fact that this occurs before the test runner outputs that tests have been executed through me for a loop.

------ Run test started ------
[xUnit.net 00:00:00.2654038]   Discovering: ....AuthZ.Tests
[xUnit.net 00:00:00.4587327]   Discovered:  ....AuthZ.Tests
[xUnit.net 00:00:00.7084239]   Discovering: ....Core.NodaTime.Tests
[xUnit.net 00:00:00.8702858]   Discovered:  ....Core.NodaTime.Tests
[xUnit.net 00:00:01.1202224]   Discovering: ....Core.Unity.Tests
[xUnit.net 00:00:01.2570637]   Discovered:  ....Core.Unity.Tests
System.AppDomainUnloadedException: Attempted to access an unloaded AppDomain. This can happen if the test(s) started a thread but did not stop it. Make sure that all the threads started by the test(s) are stopped before completion.
[xUnit.net 00:00:01.6189645]   Discovering: ....Web.OData.Tests
[xUnit.net 00:00:01.9658827]   Discovered:  ....Web.OData.Tests
[xUnit.net 00:00:02.2347184]   Discovering: ....AuthN.Tests
[xUnit.net 00:00:02.5320793]   Discovered:  ....AuthN.Tests
[xUnit.net 00:00:02.8185906]   Discovering: ....Core.Tests
[xUnit.net 00:00:03.7219920]   Discovered:  ....Core.Tests
[xUnit.net 00:00:03.9733561]   Discovering: ....Entities.Tests
[xUnit.net 00:00:04.0885750]   Discovered:  ....Entities.Tests
[xUnit.net 00:00:04.3412430]   Discovering: ....Mapper.Tests
[xUnit.net 00:00:04.7662632]   Discovered:  ....Mapper.Tests
[xUnit.net 00:00:04.9627827]   Discovering: ....Repository.MongoDB.Tests
[xUnit.net 00:00:05.0857825]   Discovered:  ....Repository.MongoDB.Tests
System.AppDomainUnloadedException: Attempted to access an unloaded AppDomain. This can happen if the test(s) started a thread but did not stop it. Make sure that all the threads started by the test(s) are stopped before completion.
[xUnit.net 00:00:05.3236843]   Discovering: ....Serializer.Json.Tests
[xUnit.net 00:00:05.4421362]   Discovered:  ....Serializer.Json.Tests
[xUnit.net 00:00:05.6906524]   Discovering: ....ServiceBus.EventHub.Tests
[xUnit.net 00:00:05.9134276]   Discovered:  ....ServiceBus.EventHub.Tests
[xUnit.net 00:00:06.1932999]   Discovering: ....Web.Tests
[xUnit.net 00:00:06.9457948]   Discovered:  ....Web.Tests
[xUnit.net 00:00:07.2512793]   Starting:    ....AuthZ.Tests
[xUnit.net 00:00:08.5344274]   Finished:    ....AuthZ.Tests
[xUnit.net 00:00:08.8198517]   Starting:    ....Core.NodaTime.Tests
...

Here are the packages in use by my unit tests:

<?xml version="1.0" encoding="utf-8"?>
<packages>
  <package id="FluentAssertions" version="4.0.0" targetFramework="net45" />
  <package id="NSubstitute" version="1.9.1.0" targetFramework="net45" />
  <package id="StyleCop.MSBuild" version="4.7.47.0" targetFramework="net45" developmentDependency="true" />
  <package id="Xbehave" version="2.0.1" targetFramework="net45" />
  <package id="Xbehave.Core" version="2.0.1" targetFramework="net45" />
  <package id="xunit" version="2.1.0" targetFramework="net45" />
  <package id="xunit.abstractions" version="2.0.0" targetFramework="net45" />
  <package id="xunit.assert" version="2.1.0" targetFramework="net45" />
  <package id="xunit.core" version="2.1.0" targetFramework="net45" />
  <package id="xunit.extensibility.core" version="2.1.0" targetFramework="net45" />
  <package id="xunit.extensibility.execution" version="2.1.0" targetFramework="net45" />
  <package id="xunit.runner.visualstudio" version="2.1.0" targetFramework="net45" />
</packages>

This seems to happen at random intermittent points in the test discovery and executing stages. This snippet from the log is happening well before the log indicates it is starting any tests. This occurs in both VSO and locally in Visual Studio. I do suspect that there is a leaked task in there somewhere in my tests, but the fact that this occurs before the test runner outputs that tests have been executed through me for a loop.

------ Run test started ------
[xUnit.net 00:00:00.2654038]   Discovering: ....AuthZ.Tests
[xUnit.net 00:00:00.4587327]   Discovered:  ....AuthZ.Tests
[xUnit.net 00:00:00.7084239]   Discovering: ....Core.NodaTime.Tests
[xUnit.net 00:00:00.8702858]   Discovered:  ....Core.NodaTime.Tests
[xUnit.net 00:00:01.1202224]   Discovering: ....Core.Unity.Tests
[xUnit.net 00:00:01.2570637]   Discovered:  ....Core.Unity.Tests
System.AppDomainUnloadedException: Attempted to access an unloaded AppDomain. This can happen if the test(s) started a thread but did not stop it. Make sure that all the threads started by the test(s) are stopped before completion.
[xUnit.net 00:00:01.6189645]   Discovering: ....Web.OData.Tests
[xUnit.net 00:00:01.9658827]   Discovered:  ....Web.OData.Tests
[xUnit.net 00:00:02.2347184]   Discovering: ....AuthN.Tests
[xUnit.net 00:00:02.5320793]   Discovered:  ....AuthN.Tests
[xUnit.net 00:00:02.8185906]   Discovering: ....Core.Tests
[xUnit.net 00:00:03.7219920]   Discovered:  ....Core.Tests
[xUnit.net 00:00:03.9733561]   Discovering: ....Entities.Tests
[xUnit.net 00:00:04.0885750]   Discovered:  ....Entities.Tests
[xUnit.net 00:00:04.3412430]   Discovering: ....Mapper.Tests
[xUnit.net 00:00:04.7662632]   Discovered:  ....Mapper.Tests
[xUnit.net 00:00:04.9627827]   Discovering: ....Repository.MongoDB.Tests
[xUnit.net 00:00:05.0857825]   Discovered:  ....Repository.MongoDB.Tests
System.AppDomainUnloadedException: Attempted to access an unloaded AppDomain. This can happen if the test(s) started a thread but did not stop it. Make sure that all the threads started by the test(s) are stopped before completion.
[xUnit.net 00:00:05.3236843]   Discovering: ....Serializer.Json.Tests
[xUnit.net 00:00:05.4421362]   Discovered:  ....Serializer.Json.Tests
[xUnit.net 00:00:05.6906524]   Discovering: ....ServiceBus.EventHub.Tests
[xUnit.net 00:00:05.9134276]   Discovered:  ....ServiceBus.EventHub.Tests
[xUnit.net 00:00:06.1932999]   Discovering: ....Web.Tests
[xUnit.net 00:00:06.9457948]   Discovered:  ....Web.Tests
[xUnit.net 00:00:07.2512793]   Starting:    ....AuthZ.Tests
[xUnit.net 00:00:08.5344274]   Finished:    ....AuthZ.Tests
[xUnit.net 00:00:08.8198517]   Starting:    ....Core.NodaTime.Tests
...

Here are the packages in use by my unit tests:

<?xml version="1.0" encoding="utf-8"?>
<packages>
  <package id="FluentAssertions" version="4.0.0" targetFramework="net45" />
  <package id="NSubstitute" version="1.9.1.0" targetFramework="net45" />
  <package id="StyleCop.MSBuild" version="4.7.47.0" targetFramework="net45" developmentDependency="true" />
  <package id="Xbehave" version="2.0.1" targetFramework="net45" />
  <package id="Xbehave.Core" version="2.0.1" targetFramework="net45" />
  <package id="xunit" version="2.1.0" targetFramework="net45" />
  <package id="xunit.abstractions" version="2.0.0" targetFramework="net45" />
  <package id="xunit.assert" version="2.1.0" targetFramework="net45" />
  <package id="xunit.core" version="2.1.0" targetFramework="net45" />
  <package id="xunit.extensibility.core" version="2.1.0" targetFramework="net45" />
  <package id="xunit.extensibility.execution" version="2.1.0" targetFramework="net45" />
  <package id="xunit.runner.visualstudio" version="2.1.0" targetFramework="net45" />
</packages>
@eriove

This comment has been minimized.

Show comment
Hide comment
@eriove

eriove Nov 3, 2015

Not sure if this helps at all. I get the same warning on TFS 12.0.21005.1 with the following Xunit packages:

<packages>
  <package id="xunit" version="2.1.0" targetFramework="net452" />
  <package id="xunit.abstractions" version="2.0.0" targetFramework="net452" />
  <package id="xunit.assert" version="2.1.0" targetFramework="net452" />
  <package id="xunit.core" version="2.1.0" targetFramework="net452" />
  <package id="xunit.extensibility.core" version="2.1.0" targetFramework="net452" />
  <package id="xunit.extensibility.execution" version="2.1.0" targetFramework="net452" />
  <package id="xunit.runner.msbuild" version="2.1.0" targetFramework="net452" />
  <package id="xunit.runner.visualstudio" version="2.1.0" targetFramework="net452" />
</packages>

It's only a test solution set up. So there is only one test looking like this:

[Fact]
public void DummyTest()
{

}

eriove commented Nov 3, 2015

Not sure if this helps at all. I get the same warning on TFS 12.0.21005.1 with the following Xunit packages:

<packages>
  <package id="xunit" version="2.1.0" targetFramework="net452" />
  <package id="xunit.abstractions" version="2.0.0" targetFramework="net452" />
  <package id="xunit.assert" version="2.1.0" targetFramework="net452" />
  <package id="xunit.core" version="2.1.0" targetFramework="net452" />
  <package id="xunit.extensibility.core" version="2.1.0" targetFramework="net452" />
  <package id="xunit.extensibility.execution" version="2.1.0" targetFramework="net452" />
  <package id="xunit.runner.msbuild" version="2.1.0" targetFramework="net452" />
  <package id="xunit.runner.visualstudio" version="2.1.0" targetFramework="net452" />
</packages>

It's only a test solution set up. So there is only one test looking like this:

[Fact]
public void DummyTest()
{

}
@Worthy7

This comment has been minimized.

Show comment
Hide comment
@Worthy7

Worthy7 Dec 7, 2016

I too can confirm that this is just a visual thing and does not affect actual test results.
I think this has something to do with threads.

Worthy7 commented Dec 7, 2016

I too can confirm that this is just a visual thing and does not affect actual test results.
I think this has something to do with threads.

@Robban1980

This comment has been minimized.

Show comment
Hide comment
@Robban1980

Robban1980 Dec 13, 2016

I can reproduce the issue when i run the Visual Studio Test Runner and there is a MSTest test in another assembly that is throwing an exception.
But as previously said i have not found any side effects from this.

I can reproduce the issue when i run the Visual Studio Test Runner and there is a MSTest test in another assembly that is throwing an exception.
But as previously said i have not found any side effects from this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment