using In Memory Database not working nicely with ASP.Net Core TestServer and Startup.cs #5150

Closed
stefanhendriks opened this Issue Apr 22, 2016 · 12 comments

Comments

Projects
None yet
5 participants
@stefanhendriks

Note: I have unit tests working with an InMemoryDatabase. This is different :)

Context:
I'm using ASP.net core (MVC) with Entity Framework Core. I have a running web app, it stores some stuff in the database properly.

Goal:
Now I want to run my same app with an In Memory Database. I want to run integration tests against this app.

So I made sure setting up the database is different in my integration tests (using a Test*Startup.cs overriding a method).

        public override void SetUpDataBaseAndMigrations(IServiceCollection services)
        {   
            services
                .AddEntityFramework()
                .AddInMemoryDatabase()
                .AddDbContext<CmsDbContext>(options => options.UseInMemoryDatabase());
        }

When booting the app, it breaks on several levels. I have posted the code below and marked the lines with comments so I can refer to them. (I have tried multiple things)

public virtual void BootstrapDatabase(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
        {
            var logger = loggerFactory.CreateLogger<Startup>();
            //Create Database
            using (var serviceScope = app.ApplicationServices.GetRequiredService<IServiceScopeFactory>()
                    .CreateScope())
            {
                var dbContext = serviceScope.ServiceProvider.GetService<CmsDbContext>();
                var databaseFacade = dbContext.Database;
                var dbConnection = databaseFacade.GetDbConnection();
                logger.LogInformation($"Using SQL Connection: {dbConnection.DataSource}");

                databaseFacade.Migrate();

                if (dbContext.CmsPages.GetRootPage() == null)
                {
                    var homepage = dbContext.SavePage<ContentPage>("home", null);
                    var contentPage = dbContext.SavePage<ContentPage>("contentPage1", homepage.Page.Id);
                    var contentPage2 = dbContext.SavePage<ContentPage>("contentPage2", homepage.Page.Id);
                    var nestedPage = dbContext.SavePage<ContentPage>("nestedPage", contentPage2.Page.Id);
                }
            }

        }

Running the app with above, results in:

No service for type 'Microsoft.Data.Entity.Storage.IRelationalConnection' has been registered.

Which is triggered by databaseFacade.GetDbConnection();. So commenting that out (and the logger statement) will let us continue. But then breaks on the next line, where it tries to run Migrate:

No service for type 'Microsoft.Data.Entity.Migrations.IMigrator' has been registered.

Ok, so perhaps all this is not needed for an InMemoryDatabase. Who needs to run migrations anyway. :)

When turning that off it will break on creating a ContentPage with:

The property 'version_type' on entity type 'ContentPage' has a temporary value while attempting to change the entity's state to 'Unchanged'. Either set a permanent value explicitly or ensure that the database is configured to generate values for this property.

This version_type is created by a modelBuilder (snippet following):

        protected override void OnModelCreating(ModelBuilder modelBuilder)
        {
            modelBuilder.Entity<CmsPageVersion>()
             .HasDiscriminator<string>("version_type");

I'm a bit confused though. I would not expect anything like this to pop-up. Especially since running a simple Unit Test and storing (and retrieving) stuff seems to work.

Running the web app with an In Memory Database - not as an integration test
To make sure it is not related to any testing framework related things. I simply changed my DB setup to use an InMemoryDatabase (as above). The results are the same as above. So it does not seem to matter if I try to run an integration test or not. It looks like some deeper down problem, or some fundamental thing I am misunderstanding.

So before I digg (much) deeper into the rabbit hole I decided to put up a question here.

I was expecting to just be able to boot the application, but apparently I am missing something (or is the InMemoryDatabase not suited for integration testing?).

Any clues would be much appreciated! Thanks!

@stefanhendriks stefanhendriks changed the title from using In Memory Database not working nicely with ASP.Net 6 TestServer and Startup.cs to using In Memory Database not working nicely with ASP.Net Core TestServer and Startup.cs Apr 25, 2016

@stefanhendriks

This comment has been minimized.

Show comment
Hide comment
@stefanhendriks

stefanhendriks Apr 25, 2016

Update:
Using Sqlite instead of InMemory works.

For now this is sufficient, but it does not solve the actual problem.

For me, I want to run integration tests and run them isolated. Using Sqlite I can do this. I can get rid of the DB files and run tests again. I also don't need a separated DB instance. Other ways would be to use a different Schema (on same DB instance).

Update:
Using Sqlite instead of InMemory works.

For now this is sufficient, but it does not solve the actual problem.

For me, I want to run integration tests and run them isolated. Using Sqlite I can do this. I can get rid of the DB files and run tests again. I also don't need a separated DB instance. Other ways would be to use a different Schema (on same DB instance).

@ajcvickers

This comment has been minimized.

Show comment
Hide comment
@ajcvickers

ajcvickers Apr 25, 2016

Member

@stefanhendriks The first two errors are, as you said, because the code is trying to use relational-specific functionality with a non-relational database. I believe there is now a better exception message for this, and not just the "no service registered" message.

The third issue looks like a dupe of #4285 which is fixed in the current code base.

Member

ajcvickers commented Apr 25, 2016

@stefanhendriks The first two errors are, as you said, because the code is trying to use relational-specific functionality with a non-relational database. I believe there is now a better exception message for this, and not just the "no service registered" message.

The third issue looks like a dupe of #4285 which is fixed in the current code base.

@rowanmiller

This comment has been minimized.

Show comment
Hide comment
@rowanmiller

rowanmiller Apr 25, 2016

Member

Just to build on @ajcvickers comments - InMemory is not intended to be a relational database. As with anything that is used with as a substitute for testing, the ability for it to replicate the behavior of the actual database you ultimately target is limited. InMemory gets you a step closer than creating test doubles for the context etc. You can also use SQLite with in-memory mode to replicate something closer to SQL Server, but ultimately even that is going to behave differently in some cases.

Member

rowanmiller commented Apr 25, 2016

Just to build on @ajcvickers comments - InMemory is not intended to be a relational database. As with anything that is used with as a substitute for testing, the ability for it to replicate the behavior of the actual database you ultimately target is limited. InMemory gets you a step closer than creating test doubles for the context etc. You can also use SQLite with in-memory mode to replicate something closer to SQL Server, but ultimately even that is going to behave differently in some cases.

@stefanhendriks

This comment has been minimized.

Show comment
Hide comment
@stefanhendriks

stefanhendriks Apr 26, 2016

Thanks for responding.

From a testing POV I would expect an InMemory database atleast to have stubbed all mechanics (dependencies), even though they would not do anything. But perhaps it should be explicitly enabled somewhere? So atleast as a tester you are making the explicit choice to use a 'fake database'. (which is a step further than "in Memory" - which you could literally just see as a DB stored in memory , not in files).

Improving on the Exception message is good. I think that will prevent confusion. However, if possible, I would suggest to just give a stubbed IRelationalConnection and what not.

In my case I would not even do much DB specific stuff in this test. I would simply visit a page (GET). And I also wanted to test if my POST worked. I noticed some binding failures earlier on and wanted to make sure it kept working (hence the integration test). Especially related to Culture settings (for parsing Datetime, etc) and validations. I don't want to run UI tests (they are too fragile).

If there are other ways to do this, that would also be fine. For now I have found a good solution:

@rowanmiller I tried your suggestion for an In memory SQLite db. And it worked like a charm, although I had to do:

dbContext.Database.OpenConnection();

open a connection before running

dbContext.Database.EnsureCreated();

I'm aware it is not the same as SQL Server, but thats also not the problem area in my test. (I'm not doing any (complicated) DB interactions).

I will wrap this up in a blog post for future reference, although things will probably change a bit in RC2.

stefanhendriks commented Apr 26, 2016

Thanks for responding.

From a testing POV I would expect an InMemory database atleast to have stubbed all mechanics (dependencies), even though they would not do anything. But perhaps it should be explicitly enabled somewhere? So atleast as a tester you are making the explicit choice to use a 'fake database'. (which is a step further than "in Memory" - which you could literally just see as a DB stored in memory , not in files).

Improving on the Exception message is good. I think that will prevent confusion. However, if possible, I would suggest to just give a stubbed IRelationalConnection and what not.

In my case I would not even do much DB specific stuff in this test. I would simply visit a page (GET). And I also wanted to test if my POST worked. I noticed some binding failures earlier on and wanted to make sure it kept working (hence the integration test). Especially related to Culture settings (for parsing Datetime, etc) and validations. I don't want to run UI tests (they are too fragile).

If there are other ways to do this, that would also be fine. For now I have found a good solution:

@rowanmiller I tried your suggestion for an In memory SQLite db. And it worked like a charm, although I had to do:

dbContext.Database.OpenConnection();

open a connection before running

dbContext.Database.EnsureCreated();

I'm aware it is not the same as SQL Server, but thats also not the problem area in my test. (I'm not doing any (complicated) DB interactions).

I will wrap this up in a blog post for future reference, although things will probably change a bit in RC2.

@rowanmiller

This comment has been minimized.

Show comment
Hide comment
@rowanmiller

rowanmiller Apr 28, 2016

Member

I think for folks that want something that behaves like a relational database, SQLite in in-memory mode is the right thing for us to recommend... as we pad out our docs I'll make sure that we make that clear. We could make InMemory look more like a relational database - but as we enable non-relational data stores it becomes less clear that it would be a good idea. For some APIs (such as Database.Migrate()) we could just no-op - that's the approach we have taken with transactions. But for methods like GetDbConnection() it would be a lot of work to stub out a fake implementation of DbConnection.

Member

rowanmiller commented Apr 28, 2016

I think for folks that want something that behaves like a relational database, SQLite in in-memory mode is the right thing for us to recommend... as we pad out our docs I'll make sure that we make that clear. We could make InMemory look more like a relational database - but as we enable non-relational data stores it becomes less clear that it would be a good idea. For some APIs (such as Database.Migrate()) we could just no-op - that's the approach we have taken with transactions. But for methods like GetDbConnection() it would be a lot of work to stub out a fake implementation of DbConnection.

@stefanhendriks

This comment has been minimized.

Show comment
Hide comment
@RehanSaeed

This comment has been minimized.

Show comment
Hide comment
@RehanSaeed

RehanSaeed Sep 13, 2016

Initially, I tried using WebHostBuilder.ConfigureServices, however this occurs before the Startup.ConfigureServices method, thus rendering it useless in most cases.

@stefanhendriks Your idea of overriding the Startup class is a good workaround but I'm finding that all of my controllers have gone missing from the services container. This is probably because my TestStartup class which inherits from Startup lives in my test assembly and not in the application assembly. Any good quick workarounds for this?

Initially, I tried using WebHostBuilder.ConfigureServices, however this occurs before the Startup.ConfigureServices method, thus rendering it useless in most cases.

@stefanhendriks Your idea of overriding the Startup class is a good workaround but I'm finding that all of my controllers have gone missing from the services container. This is probably because my TestStartup class which inherits from Startup lives in my test assembly and not in the application assembly. Any good quick workarounds for this?

@stefanhendriks

This comment has been minimized.

Show comment
Hide comment
@stefanhendriks

stefanhendriks Sep 13, 2016

@RehanSaeed I also have my TestStartup class in my Test assembly. Perhaps you are overriding too much? Ie, for me I have something like this in my startup.cs:

public void ConfigureServices(IServiceCollection services)
{
...
AddFrameworkDependencies(services);
...
services.AddMvc()
...
}

and in TestPortal.cs I only inject the 'different' dependencies:

public override void AddFrameworkDependencies(IServiceCollection services)
... override only singletons I want to 'stub'
}

As you can see I do this before calling the services.AddMvc() and I do not stub the entire ConfigureServices method, just a portion of it.

@RehanSaeed I also have my TestStartup class in my Test assembly. Perhaps you are overriding too much? Ie, for me I have something like this in my startup.cs:

public void ConfigureServices(IServiceCollection services)
{
...
AddFrameworkDependencies(services);
...
services.AddMvc()
...
}

and in TestPortal.cs I only inject the 'different' dependencies:

public override void AddFrameworkDependencies(IServiceCollection services)
... override only singletons I want to 'stub'
}

As you can see I do this before calling the services.AddMvc() and I do not stub the entire ConfigureServices method, just a portion of it.

@RehanSaeed

This comment has been minimized.

Show comment
Hide comment
@RehanSaeed

RehanSaeed Sep 13, 2016

Solved the missing controller issue. The test project also requires the preserveCompilationContext property set to true in project.json.

{
    "buildOptions": {
        "preserveCompilationContext": true
    }
}

Solved the missing controller issue. The test project also requires the preserveCompilationContext property set to true in project.json.

{
    "buildOptions": {
        "preserveCompilationContext": true
    }
}
@twilliamsgsnetx

This comment has been minimized.

Show comment
Hide comment
@twilliamsgsnetx

twilliamsgsnetx Sep 27, 2016

@stefanhendriks I just want to say thank you. I was using sqlite in memory for a testing database, but I couldn't figure out for the life of me why it was not creating the tables. I had EnsureCreated()... but what I failed to do was open the connection first.

@stefanhendriks I just want to say thank you. I was using sqlite in memory for a testing database, but I couldn't figure out for the life of me why it was not creating the tables. I had EnsureCreated()... but what I failed to do was open the connection first.

@stefanhendriks

This comment has been minimized.

Show comment
Hide comment
@stefanhendriks

stefanhendriks Sep 28, 2016

@twilliamsgsnetx glad the blog post helped you out with EnsureCreated and such. All in all took me a while to figure out, glad that summarizing in one post helped at least one person! 💃

@twilliamsgsnetx glad the blog post helped you out with EnsureCreated and such. All in all took me a while to figure out, glad that summarizing in one post helped at least one person! 💃

@borisdj borisdj referenced this issue in borisdj/EFCore.BulkExtensions Nov 29, 2017

Closed

Does it support the InMemory provider? #24

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment