Assembly & Class Cleanup should be called even if Initialize fails

[andrewphamvk]

Description

AssemblyCleanup/ClassCleanup method is not called if AssemblyInitialize/ClassInitialize method fails.
It would make sense to have the ability to cleanup partially initialized tests runs just like TestInitialize/TestCleanup.

See below comment regarding previous decision to break compatibility with MSTestV1 and support execution of TestCleanup even after TestInitialize failure.

@arnonax : Thanks for making this explicit. This was a conscious compat break to give unit test writers a choice to cleanup partially initialized methods. However in order to make the transition from v1 extremely smooth, it does make sense to have some setting to preserve compat.

/cc: @harshjain2 , @pvlakshm

Originally posted by @AbhitejJohn in #250 (comment)

Steps to reproduce

AssemblyInitialize/AssemblyCleanup

[AssemblyInitialize]
public static void AssemblyInitialize(TestContext context)
{
    File.AppendAllLines(@"C:\MSTest.txt", new [] { "AssemblyInit." });
    throw new Exception();
}

[TestMethod]
public void TestMethod1()
{
}

[AssemblyCleanup]
public static void AssemblyCleanup()
{
    File.AppendAllLines(@"C:\MSTest.txt", new [] { "AssemblyCleanup." });
}

Output

AssemblyInit.

ClassInitialize/ClassCleanup

[ClassInitialize]
public static void ClassInitialize(TestContext context)
{
    File.AppendAllLines(@"S:\MSTest.txt", new[] { "ClassInit." });
    throw new Exception();
}

[TestMethod]
public void TestMethod1()
{
}

[ClassCleanup]
public static void ClassCleanup()
{
    File.AppendAllLines(@"S:\MSTest.txt", new[] { "ClassCleanup." });
}

Output

ClassInit.

TestInitialize/TestCleanup

[TestInitialize]
public static void TestInitialize(TestContext context)
{
    File.AppendAllLines(@"S:\MSTest.txt", new[] { "TestInit." });
    throw new Exception();
}

[TestMethod]
public void TestMethod1()
{
}

[TestCleanup]
public void TestCleanup()
{
    File.AppendAllLines(@"S:\MSTest.txt", new[] { "TestCleanup." });
}

Output

TestInit.
TestCleanup.

Expected behavior

The AssemblyCleanup and ClassCleanup methods should have a chance to run.

Actual behavior

The initialization methods throw and their corresponding cleanup method is not run.

[nohwnd]

@andrewphamvk makes sense, adding for consideration to the next sprint. 🙂