Parallelizeable tests sometimes shares memory

[zlepper]

Sometimes when unit tests are run in in parallel using the Parallelizable attribute, they accidently end up sharing memory. Which results in very change failures and disposed objects all over the place. See the attached project for a reproduction.

In rider the error list looks roughly like this:

However which tests are failing, and how many a mostly random. However the amount i have there is pretty good at reproducing the problem. No problem is observed if the Parallelizeable isn't used.

service-provider-dispose-test.zip

[jnm2]

@zlepper This addressed in the upcoming v3.13 by #2574. You will have the option to add an attribute [FixtureLifeCycle(LifeCycle.InstancePerTestCase)] to each test assembly or test class.

Until then, you'll have to refrain from using class fields and SetUp/TearDown with test-case-level parallelism because they are incompatible in 3.12 and earlier. What I've personally been doing is making all my test classes static and introducing a new private nested fixture class or helper method to handle setup. If teardown is needed, I make the nested class disposable and start each test with using var fixture = new Fixture(...);. I've found that fixture.AssertX(); etc is a nice way to minimize test boilerplate.

[zlepper]

@jnm2 Nice! Is it possible to apply it to a base class and have it work down through the inheritance heirachy? For example we use a base class for almost all our tests that takes care of initializing a dependency injector and creating and entity framework database. That's where i added the dispose code today, and encountered this issue. So i would love to apply that attribute to the base class, and have it automatically work everywhere in all the test assemblies.

If not, no biggie, we don't use parallization as much as we might be able to, I'm just curious :)

[jnm2]

I'm not sure off the top of my head, but I would honestly recommend using [assembly: FixtureLifeCycle(LifeCycle.InstancePerTestCase)] in every test project and opt out per test fixture as necessary. That's what I do with test-case-level parallelism as well.