Unable to load/reference appsettings.json when integration testing an ASP.NET Core WebApi app using MVC.Testing

[PureKrome]

Hi there 👋

I'm trying to do a basic integration test for a very simple ASP.NET Core WebApi project.

I'm leveraging the WebApplicationFactory class and when I do this, my appsettings.json file located in my test project is not getting used. The appsettings.development.json file in the WebApi project (the SUT) is getting used.

I've tried to use Source Link/Source Debugging using the MS Symbol Server to step through the WebApplicationFactory class to figure out why/what. It's a bit hard to grok.

I even sorta tried to use the TEST_CONTENTROOT_APPNAME trick/hack I noticed in the source code but I couldn't get that to work properly (I think i'm not correctly adding this via my custom protected override IWebHostBuilder CreateWebHostBuilder() method early on in the bootstrapping).

Anyways -> can anyone (repo members?) please provide some clues to how I can use my own appsettings.json locally in my test project ... and not the ones in the SUT.

Why?

SUT appsettings.development.json : db points to localhost
Test project appsettings.json : db will point to some 'dev' server in the cloud.

[poke]

Instead of changing the content root, which will probably also have lots of other side effects you generally want to avoid, you can just reconfigure the application you run with the WebApplicationFactory.

That’s actually pretty easy to do:

public class TestApplicationFactory : WebApplicationFactory<Startup>
{
    protected override void ConfigureWebHost(IWebHostBuilder builder)
    {
        builder.ConfigureAppConfiguration(config =>
        {
            var integrationConfig = new ConfigurationBuilder()
                .AddJsonFile("integrationsettings.json")
                .Build();

            config.AddConfiguration(integrationConfig);
        });
    }
}

This creates a custom WebApplicationFactory that will add a custom integration config on top of the configuration that is already used by the web application. So you can use this nicely to adjust configuration for test purposes, e.g. change the connection string for your database or something like that.

Because this uses a separate ConfigurationBuilder, that builder is not restricted to the base path that is set to the content root with the host builder. So you can configure the additional configuration completely independently.

In my example above, I add a integrationsettings.json file that is included in the project. You will need to configure this to be copied to the output directory on build. You can configure that with Visual Studio, or by modifying the .csproj of your test project to include this:

<ItemGroup>
    <Content Include="integrationsettings.json" CopyToOutputDirectory="PreserveNewest" />
</ItemGroup>

You could even add other sources for your integration tests. For example, in some of my projects I also add environment variables with a custom TEST_ prefix so that I can set that in my CI. Or you could add user secrets and configure user secrets separately for your test project.

Bottom line is that you don’t need to modify the default behavior for this. And instead of completely replacing the configuration (although you could do that using above approach), it’s much easier to simply add test-specific configuration on top, so that you just adjust a few configurations instead of having to repeat everything even if it would be the same for the test. After all, you want to test your application as closely to production as possible, so sharing the configuration is actually desired.

[PureKrome]

@poke Thank's heaps for the reply. Interesting ... :) I actually have this custom file and I was also trying to do the same thing (but I wasn't doing something right). I definitely had my appsettings.json file copied to the output directory, so I was doing something wrong.

What I was unsure about was this bit:

Because this uses a separate ConfigurationBuilder, that builder is not restricted to the base path that is set to the content root with the host builder. So you can configure the additional configuration completely independently.

So are you suggesting, that ... creating a new ConfigurationBuilder and reading in config from my test project appsettings.json (or in your example, integrationsettings.json) .. this will:

a) Add to the existing settings and if those settings already exist, overwrite them [Add or Update / Upsert]
b) Overwrite all the existing settings (from the SUT) and just use these ones? [Overwrite]

[poke]

In my example, the configuration is added to the configuration set from the application. The default application configuration sources (appsettings.json, appsettings.<Environment>.json, environment variables, user secrets) are kept and the additional integration configuration is added after that. So you can use it to overwrite existing values.

The ConfigurationBuilder in my example is actually only used so that you can add a JSON file that does not depend on the base path that is configured with the application’s configuration builder (the default one).

That being said, you could skip the normal application configuration completely if you modified the configuration builder that gets passed to ConfigureAppConfiguration and removed the already configured configuration builders there. But as I said, in general you want to make use of the existing application configuration and just update a few values for integration test purposes.

So adding another source on top of it with just a few configuration values is actually a pretty good approach.

[PureKrome]

Thanks again @poke for the helpful reply. Just one further questions relating to this. If, for example, you have a specific DB connection string in your integrationsettings.json file .. which then overwrites the existing setting loaded from the SUT ... how would you use that setting in the actual TEST method?

for example:

[Fact]
public async Task GivenFoo_MethodAsync_ReturnsBaa()
{
    // Arrange.
    transaction.Start(); // This is just some lame pseduo code.

    // Assert.
    using (var db = new SqlConnection(connectionString))
    {
        var result = "SELECT .... "; // Make sure the item was added to the DB.
    }

    transaction.RollBack(); // you get the idea.
}

is there some way to access the IConfigurationRoot in your test OR somewhere else in your custom TestApplicationFactory class which u can then expose via inheritance?

[poke]

The usual way would be using the WebApplicationFactory for integration tests; so something like this:

public class MyTests : IClassFixture<TestApplicationFactory>
{
    private readonly TestApplicationFactory _factory;
    public MyTests(TestApplicationFactory factory)
    {
        _factory = factory;
    }

    [Fact]
    public async Task ExampleTest()
    {
        // arrange
        var client = _factory.CreateClient();

        // act
        var result = await client.GetAsync("/api/foo");

        // assert
        result.EnsureSuccessStatusCode();
        var textContent = await result.Content.ReadAsString();
        Assert.Equal("{\"foo\": \"bar\"}", textContent);
    }
}

So since you are using the TestApplicationFactory, this will launch your web application for this test, using that integrationsettings.json we configured, and provide a HttpClient for you to use to send requests to your web application. So you won’t have to actually access your database directly that way.

That being said, what I often do is have a mutating endpoint I call and then I want to verify that the change made it into the database. For that, we can resolve services from the factory directly:

[Fact]
public async Task CreateFoo()
{
    // arrange
    var client = _factory.CreateClient();

    // act
    var result = client.PostAsJsonAsync("/foo", new { id = 4, bar = "baz" });

    // assert
    result.EnsureSuccessStatusCode();

    using (var scope = _factory.Server.ApplicationServices.CreateScope())
    {
        var db = scope.ServiceProvider.GetService<ApplicationDbContext>();

        var foo = await db.Foos.FirstOrDefaultAsync(f => f.Id == 4);
        Assert.NotNull(foo);
        Assert.Equal("baz", foo.Bar);
    }
}

In addition, if you don’t want to run full integration tests against your web application but just against a real database (for example for database related tests), then what I additionally do is extract that integrationConfig from above into a utility method that I can then call from either the TestApplicationFactory or from other tests. So I can use that configuration to retrieve the connection string manually in case I need to.

[PureKrome]

That being said, what I often do is have a mutating endpoint I call and then I want to verify that the change made it into the database.

Yep - this is the example I was trying to explain (but did a poor job of it).

var db = scope.ServiceProvider.GetService<ApplicationDbContext>();

In my case, we're not using something like EF (i'm assuming this might be EF or Cosmos or whatever). We use Dapper to do all our DB stuff. So instead of passing around a DBContext we pass around a ConnectionString (well, technically we pass around a DatabaseSettings POCO which contains this. So if some class does DB stuff, then it will take an instance of a DatabaseSettings POCO.)

Also in your test above, that POST will mutate/change the DB. Why aren't you rolling that back? Or you usually would but this is just some quick and simple example code?

extract that integrationConfig from above into a utility method that I can then call from either the TestApplicationFactory or from other tests. So I can use that configuration to retrieve the connection string manually in case I need to.

So do you mean something like this?

public class TestWebApplicationFactory : WebApplicationFactory<Startup>
{
    private IConfigurationRoot _configurationRoot;

    public IConfigurationRoot ConfigurationRoot
    {
        get
        {
            return _configurationRoot ?? (_configurationRoot = new ConfigurationBuilder()
            .SetBasePath(Directory.GetCurrentDirectory())
            .AddJsonFile("integrationsettings.json", optional: true)
            .Build());
        }
    }

    public string SqlConnectionString
    {
        get { return ConfigurationRoot.GetValue<string>("SqlServer:ConnectionString"); }
    }

    protected override void ConfigureWebHost(IWebHostBuilder builder)
    {
        builder.UseEnvironment(EnvironmentName.Development);

        builder.ConfigureAppConfiguration(config =>
        {
            var integrationConfig = new ConfigurationBuilder()
                .AddJsonFile("integrationsettings.json")
                .Build();

            config.AddConfiguration(integrationConfig);
        });
    }
}

where it does:

1. manually ADDS the integrationsettings.json to the existing app-settings for the SUT
2. Has a helper/utility property which (again) reads the integrationsettings.json file into a local variable and references data off that (in this case, `SqlServer:ConnectionString')

like this?

[poke]

So instead of passing around a DBContext we pass around a ConnectionString

Then you can just inject the connection string POCO there. You have the full DI container, so do whatever you need to do ;)

Also in your test above, that POST will mutate/change the DB. Why aren't you rolling that back? Or you usually would but this is just some quick and simple example code?

This is an integration test on a separate database which will be created only for the test execution and dropped completely afterwards. So I don’t really care about left over stuff, as long as it does not conflict with other tests. And I try to design my tests in a way that they don’t. But of course, that’s totally up to you.

So do you mean something like this?

Actually, I was thinking more about something like this:

public static class TestHelper
{
    public IConfiguration GetTestConfiguration()
        =>  new ConfigurationBuilder()
            .AddJsonFile("integrationsettings.json")
            .Build();
}

// and then inside of the TestApplicationFactory
builder.ConfigureAppConfiguration(config =>
{
    config.AddConfiguration(TestHelper.GetTestConfiguration());
});

That way, I do not need to depend on the full web application factory (which means that it spawns the whole application for integration tests) and can just use the config directly in a less-than-full integration test.

So inside of another test, that is not a full integration test, you could do it like this:

[Fact]
public Task AddItem()
{
    // arrange
    var connectionString = TestHelper.GetTestConfiguration().GetConnectionString("DefaultConnection");
    var service = new ExampleService(connectionString);

    // act
    service.Add(new Item { Foo = "Bar" });

    // assert
    using (var db = new SqlConnection(connectionString))
    {
        // verify stuff
    }
}

[mkArtakMSFT]

Thanks for contacting us. We believe that the question you've raised have been answered. If you still feel a need to continue the discussion, feel free to reopen it and add your comments.

[PureKrome]

Thanks for the help! Much appreciated. Yep, question has been answered.