Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
IErrorMessage doesn't give context on failure #75
I'm not sure how to handle
Not every failure has a straight line to a test. There are a LOT of things that happen on the way to getting something to run that can't be directly attributed to any one particular test case. Theoretically, this is a "oh god, the world fell apart" message, which means that having "nice" handling for one probably isn't necessary.
I'd be tempted to toss up some catastrophic failure dialog box in response to this.
They're not catastrophic errors - they can be raised multiple times and they don't stop execution. And perhaps more importantly, it's (usually) user code that's causing them to fail. Specifically, test class and collection fixtures throwing in Dispose (and ctor for xunit1). As a user, I'd expect the test framework to gracefully handle exceptions from my code.
Currently, if a fixture's constructor throws, it's reported as part of the test result (and the test is still run!?) but if a fixture's Dispose throws, it's reported as a catastrophic error. Similarly, the change I made to the xunit1 runner should have been a
Surprisingly, the only genuine catastrophic error that should still raise an
Should there be