Skip to content

MSpec Resharper Test Runner is calling IAssemblyContext once per test class #17

Closed
ewilde opened this Issue Jul 12, 2010 · 17 comments

7 participants

@ewilde
ewilde commented Jul 12, 2010

When running mspec tests using the resharper test runner IAssemblyContext.OnAssemblyStart and IAssemblyContext.OnAssemblyComplete get called once per test.

I repro c# project of this issue can be downloaded here:
http://jobscalp.googlecode.com/files/mspecbug.zip

If you run the above code in the Resharper test runner it will fail, run it using mspec.exe and it will pass. I'm using VS2010 and Resharper 5.1

Ed.

@agross
machine member
agross commented Jul 12, 2010

The behavior you describe is because the ReSharper runner will run each context isolated from the other contexts using RunMember. Basically, there's no way to tell from the ReSharper runner if there are more contexts to be run or if the current context is the last one.

Running specs differently (through RunAssembly etc.) poses a bit of a problem since you can filter down/select/delete contexts in the ReSharper Unit Test Session window. The information about which contexts are selected/need to run cannot be translated into a call to RunAssembly (which handles IAssembyContext properly in terms of once-per-assembly).

@agross
machine member
agross commented Jul 18, 2010

Is this an issue for you in production or can you live with the fact that test runs with the ReSharper runner will create assembly contexts for each context. From my point of view, it should not matter if you run once or many contexts inside an IAssemblyContext.

@ewilde
ewilde commented Jul 18, 2010

I am using IAssemblyContext to launch and tear down a UI, which is an expensive operation.

Each context then automates the UI and expects it to already exist and have been initialized.

We can live with the current behaviour, just thought the difference in behaviour between resharper and the commandline runner is confusing to the developer.

@chrisnicola

Ran into this one myself. I wonder if there isn't something that can be done about this. Perhaps when the resharper SDK is out this can be looked into further. I'm using the OnAssemblyStart/Complete to setup and cleanup a mongodb. It's lightweight so I don't really mind using it for some quick and dirty integration tests.

If it is going to be trouble though I may just use NUnit or something for these integration tests, it isn't that big a deal.

@nmosafi
nmosafi commented Aug 31, 2011

I had the same issue, decided to use a static initaliser or static lazy initialisation for doing startup stuff, and the AppDomain.CurrentDomain.DomainUnload event to cleanup. Seems to work ok fairly consistently

@agross
machine member
agross commented Aug 31, 2011

Neil, can you please post an example?

@nmosafi
nmosafi commented Aug 31, 2011

Sure, here's a little thing I wrote for my WatiN tests so they can lazily open a shared browser when the first test runs, and will then close it when they finish running

public class BrowserAccess
{
    private static Browser _browser;

    static BrowserAccess()
    {
        AppDomain.CurrentDomain.DomainUnload += HandleDomainUnload;
    }

    private static void HandleDomainUnload(object sender, EventArgs e)
    {
        if (_browser != null)
        {
            _browser.Close();
        }
    }

    public static Browser Browser
    {
        get
        {
            if (_browser == null)
            {
                _browser = new IE(); 
                _browser.BringToFront();
            }
            return _browser;
        }
    }
}
@AnthonyMastrean
machine member

@ewilde, those sound like heavy duty tests, integration or acceptance, at least. Can you run them from the command line mspec runner? Do you need to be running those out of your IDE? If it helps, I've published the MSpec tools for the latest release as a Chocolaty package (so they go in your Windows path and can be called from any console).

@nmosafi
nmosafi commented Jan 9, 2012

Why would you want any tests running outside of your IDE?

@AnthonyMastrean
machine member

@nmosafi The IDE is a slow dinosaur! And apparently, the problem is by design from the ReSharper team. I'm giving you a workaround. Why do you hate the cmd line ;)

@agross
machine member
agross commented Jan 9, 2012
@AlistairB

I have essentially the opposite problem. I have put nhibernate code that needs to be run before and after each test (http://stackoverflow.com/questions/5906174/mspec-run-generic-setup-and-tear-down-code-with-each-test) which was working great in Resharper, but doesn't work in my CI environment.

Is there currently a way to have code run before and after all every test that will work in Resharper and via the command line?

@agross
machine member
@nmosafi
nmosafi commented Jan 10, 2012

I'm not against the command line, but also like a good IDE. The I in IDE is the key, and command line isn't an integrated experience. Having the ability to run specific tests from within the code, rerun failed tests, debug tests, quickly navigate to stacktraces etc, is important.

@agross agross added a commit to agross/machine.specifications that referenced this issue Feb 1, 2013
@agross agross Ensuring IAssemblyContexts are executed only once, even when running …
…several contexts using one instance of the AssemblyRunner. Closes #17
fba25c2
@agross agross closed this in 21d0534 Feb 13, 2013
@nicklasisraelsson

I still get this issue when using resharper as testrunner even when using the latest code.

@agross
machine member
agross commented Feb 19, 2013

Can you please make extra sure that mspec.dll is 0.5.11 in both the ReSharper plugins and the bin directory of you application?

@nicklasisraelsson

Sorry about that, you were right. I hadn't updated the assemblies in the resharper plugin. Now it works. Thanks a bunch!

@nicklasisraelsson nicklasisraelsson referenced this issue in machine-visualstudio/machine.vstestadapter Feb 20, 2013
Closed

ApplicationContext is run one time per test class #3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.