ConfigurationManager doesn't find config file with "dotnet test"

[gboucher90]

Hello,

When run with "dotnet test" and NUnit, the actual entry point is the "testhost":

.nuget/packages/microsoft.testplatform.testhost/15.0.0/lib/netstandard1.5/testhost.dll

Then it is looking at the wrong place and with wrong name.

Program.cs

using System;
using System.Configuration;
using NUnit.Framework;

namespace test_run
{
	[TestFixture]
    class Program
    {
		/*
        static void Main(string[] args)
        {
			ShowAppSettings();
        }*/
		
		[Test]
		public void ShowAppSettings(){
			var appSettings = ConfigurationManager.AppSettings;

			if (appSettings.Count == 0)
			{
				Console.WriteLine("AppSettings is empty.");
			}
			else
			{
				foreach (var key in appSettings.AllKeys)
				{
					Console.WriteLine("Key: {0} Value: {1}", key, appSettings[key]);
				}
			}
			
			Assert.AreEqual(2, appSettings.Count);
		}
    }	
}

App.config

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <appSettings>
    <add key="Setting1" value="May 5, 2014"/>
    <add key="Setting2" value="May 6, 2014"/>
  </appSettings>
</configuration>

csproj

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp2.0</TargetFramework>
  </PropertyGroup>
 <ItemGroup>
   <PackageReference Include="System.Configuration.ConfigurationManager" Version="4.4.0-preview2-25405-01" />
    <None Update="App.config">
      <CopyToPublishDirectory>PreserveNewest</CopyToPublishDirectory>
    </None>
	<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.0.0" />
    <PackageReference Include="NUnit" Version="3.7.1" />
    <PackageReference Include="NUnit3TestAdapter" Version="3.8.0-alpha1" />
  </ItemGroup>
</Project>

EDIT: @karelz fixed code formatting

[gboucher90]

Should we override the entry assembly in NUnit Test Adapter?
https://github.com/NeutroniumCore/Neutronium/blob/master/Tests/Infra/Tests.Infra.HTMLEngineTesterHelper/Window/AssemblyHelper.cs

[jnm2]

The app.config in effect is testhost.dll.config, not MyTests.dll.config which @gboucher90 needs.
With .NET Framework the NUnit adapter can fix this by running the test assembly in a new AppDomain with AppDomainSetup.ConfigurationFile set to MyTests.dll.config, but what's the equivalent for .NET Core? If there is no equivalent, what can @gboucher90 do to work around this?

[karelz]

I still don't understand the problem / ask here: Is there need to share the app.config between normal execution and NUnit execution? Is that the underlying problem?
Who named it 'MyTests.dl.config"? That does not look like "normal execution name". Can it be renamed?

[jnm2]

@karelz The author wants to load his app.config while testing, for any of a number of possible reasons. Maybe that's the only way to initialize a component needed for the test, or maybe there are settings that need to be validated. In any case, consider a test project named MyTests. When you build, MyTests.dll and MyTests.dll.config appear in bin\Debug.

When you run VSTest, the entry assembly is testhost.dll. ConfigurationManager knows nothing about the NUnit VSTest adapter or VSTest itself and therefore it tries to load the entry assembly's config as usual and loads testhost.dll.config from the folder which contains testhost.dll rather than loading MyTests.dll.config from the folder which contains MyTests.dll.

That means the test will fail because the configuration it directly or indirectly relies on is unavailable.

[gboucher90]

Yes it is exactly what @jnm2 explained.

I have many tests (mainly integration ones) which relies on App.config data. Using .Net Framework NUnit is able to create a new app domain and specify the entry point. The later is used by System.Configuration.ConfigurationManager to find the correct App.config file.

This is not possible anymore and the code under test fails to find the configuration.

[karelz]

Can your test projects copy the app.config into the "correct one"?
Can you open config file explicitly using ConfigurationManager.OpenExeConfiguration in your app?

[jnm2]

Can your test projects copy the app.config into the "correct one"?

I don't think so. This would mean overwriting the testhost.exe.config that ships with VSTest in the dotnet sdk path.

[karelz]

I would expect the entry assembly to be in the output directory (maybe copied from somewhere). Is that not the case?

[jnm2]

@karelz I haven't observed the copying of a test host. This project does not copy a test host to the output directory:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netcoreapp2.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.5.0" />
    <PackageReference Include="NUnit" Version="3.9.0" />
    <PackageReference Include="NUnit3TestAdapter" Version="3.9.0" />
  </ItemGroup>

</Project>

It copies MyTests.dll and NUnit3.TestAdapter.dll with its dependencies Mono.Cecil.dll and nunit.engine.netstandard.dll.

Running from Test Explorer

Environment.CommandLine: C:\Users\Joseph\.nuget\packages\microsoft.testplatform.testhost\15.5.0\lib\netstandard1.5\testhost.dll --port 59436 --endpoint 127.0.0.1:059436 --role client --parentprocessid 20316 --telemetryoptedin true

Assembly.GetEntryAssembly().Location: C:\Users\Joseph\.nuget\packages\microsoft.testplatform.testhost\15.5.0\lib\netstandard1.5\testhost.dll

Running via dotnet test

Environment.CommandLine: C:\Users\Joseph\.nuget\packages\microsoft.testplatform.testhost\15.5.0\lib\netstandard1.5\testhost.dll --port 59496 --endpoint 127.0.0.1:059496 --role client --parentprocessid 11844 --telemetryoptedin false

Assembly.GetEntryAssembly().Location: C:\Users\Joseph\.nuget\packages\microsoft.testplatform.testhost\15.5.0\lib\netstandard1.5\testhost.dll

[danmoseley]

@JeremyKuhne thoughts on this?

[okwolf]

We're facing this issue too and it's a blocker for our .net core migration. The easiest way to check is to have this inside your test code

var myConfig = ConfigurationManager.OpenExeConfiguration(ConfigurationUserLevel.None).FilePath;

And you'll see that instead of referencing C:\myapp\bin\Debug\netcoreapp2.0\myapp.dll.config it references C:\myapp\bin\Debug\netcoreapp2.0\testhost.dll.config

What is the intended workaround? What about a patched release?

[SidShetye]

+1 ... also seeing this, following for a fix date.

EDIT: Our temporary workaround involved adding this to the test project's *.csproj file which copies the app.config (you should already have this) to testhost.dll.config

<Target Name="CopyCustomContent" AfterTargets="AfterBuild">
  <Copy SourceFiles="app.config" DestinationFiles="$(OutDir)\testhost.dll.config" />
</Target>

[JeremyKuhne]

The intended workaround for this sort of scenario is that you load data from a specific path. ConfigurationManager.OpenMappedExeConfiguration() will let you open any configuration file you want. Is there a reason that won't work for you?

[SidShetye]

@JeremyKuhne User level code changes to address behavioral differences between .NET framework + nunit vs .NET core + nunit smells like a kludge. Same concern but to a lesser extent on changes to the build system to account for those differences .NET F/W vs .NET Core differences.

I'm new to how vstest and testhost interplay but if they are inserting themselves in the middle, shouldn't they be responsible for maintaining transparency about this abstraction from a user code perspective? Perhaps testhost can itself call ConfigurationManager.OpenMappedExeConfiguration() to fix this ?

[JeremyKuhne]

User level code changes to address behavioral differences between .NET framework + nunit vs .NET core + nunit smells like a kludge.

Fundamentally you're kind of stuck with kludges in this case. Configuration files were tied with the AppDomain concept, which you don't have in Core. The only way to override the default path is to create a new AppDomain- setting it after creation would fundamentally break ConfigurationManager's AppSettings property (or GetSection()) as the source location would no longer be invariant. Even if we did provide a way to change it and break the invariance, that code would be unique to Core. Such an API doesn't exist on desktop.

The only way you can get consistent behavior without kludges across the platforms if you depend on custom config file locations is to explicitly open your Configuration object via OpenMappedExeConfiguration() and get AppSettings from that object, rather than the default one ConfigurationManager creates internally.

[gboucher90]

Using "OpenMappedExeConfiguration()" is not really an acceptable solution for us. We have many utility libraires referenced by applications and using directly the ConfigurationManager. Thus we cannot hardcode a specific file easily. We would need to inject everywhere the application config file name or the configuration object directly (granted that should have been done in the first place).

Anyway we are working on refactoring the code but it is quite costly and it delays the migration to .net core since we cannot run a lot of the test projects (mostly integration ones).

We may end up copying the app config file under "testhost.dll.config" but that's something I would really prefer to avoid.

[jnm2]

We may end up copying the app config file under "testhost.dll.config" but that's something I would really prefer to avoid.

Especially because that file location is user-account-global state.

[JeremyKuhne]

Sorry there isn't an easy answer. :/ Since there is only effectively one AppDomain (one set of statics) one potential way to address this would be for the test infrastructure to copy (or hardlink) the testhost.dll local to the tests to pick up specified configs (by renaming them).

[SidShetye]

@JeremyKuhne is there a new recommended approach for application settings and custom configuration for .net core apps? We’re simply going with app.config/*.config just because of existing code but open to recommendations since my team is already porting ...

[JeremyKuhne]

You can use ConfigurationManager, of course, but it is a bit heavy. If you do use the config system, using a Configuration instance is, for obvious reasons, recommended.

Outside of that, using Json seems to be the prevailing trend. It is easier to comprehend and much lighter weight than System.Configuration.

[MichaCo]

Just for the record, I'm using XUnit and having the same problems.

The workaround with copy into testhost.dll.config seem to work though, but is not really "friendly" ;)

[JeremyKuhne]

Closing. As stated, the recommendation here is to explicitly construct a Configuration object via OpenMappedExeConfiguration(). One can copy into testhost.dll.config as mentioned if in a pinch.

[obayit]

I would like to see this issue solved in a batter way. One of the main goals of integration testing is to insure that the Data Access Layer is working properly. And we all use the appsettings.json to store the connection string.
Then how can I test my business logic residing in Data Access Layer?

[jamescrowley]

Just noting that this is a significant block to our .NET Core migration too - yes, we can rewrite and refactor, but it's essentially another breaking change to deal with.

I've tried the testhost.dll.config hack but as far as I can see that has to go to a path outside the whole project directory (C:\Users\XXX.nuget\packages\microsoft.testplatform.testhost\15.3.0\lib\netstandard1.5\testhost.dll) in my case, rather than the OutDir as suggested by @SidShetye (unless I'm missing something?) - which turns a cludge into something I wouldn't want to rely on at all?

[keyrad-abasi]

Just give this one a try, it worked for me:
var config = ConfigurationManager.OpenExeConfiguration(".nuget/packages/microsoft.testplatform.testhost/15.0.0/lib/netstandard1.5/testhost.dll");

[danmoseley]

https://github.com/dotnet/corefx/issues/32095 created to find another solution.

[nsmithdev]

+1 ... also seeing this, following for a fix date.

EDIT: Our temporary workaround involved adding this to the test project's *.csproj file which copies the app.config (you should already have this) to testhost.dll.config

<Target Name="CopyCustomContent" AfterTargets="AfterBuild">
  <Copy SourceFiles="app.config" DestinationFiles="$(OutDir)\testhost.dll.config" />
</Target>

With dotnet 3.1 I'm having to reference testhost.x86.dll.config

<Target Name="CopyCustomContent" AfterTargets="AfterBuild">
    <Copy SourceFiles="App.config" DestinationFiles="$(OutDir)\testhost.x86.dll.config" />
</Target>

The statement below was very helpful for verifying the config path used

ConfigurationManager.OpenExeConfiguration(ConfigurationUserLevel.None).FilePath;

[robertmclaws]

I was noticing that there were some issues with the .NET Core 3.1 SDK not being found after the VS Preview install, and I had to install the x86 SDK manually. I'm not sure how all that got changed, but it would be nice if the runtime was using the 64-bit SDK again the way it's supposed to be.

[Daniel-Khodabakhsh]

Here's a snippet that should work for both x86 and x64 cases.
Place this in your unit test global setup:

#if NETCOREAPP
using System.Configuration;
using System.IO;
using System.Reflection;
#endif

...

// In your global setup:
#if NETCOREAPP
    string configFile = $"{Assembly.GetExecutingAssembly().Location}.config";
    string outputConfigFile = ConfigurationManager.OpenExeConfiguration(ConfigurationUserLevel.None).FilePath;
    File.Copy(configFile, outputConfigFile, true);
#endif