Let continueAfterFailure default to YES

[0xced]

Setting the continueAfterFailure property to NO is very problematic when using the XCTest asynchronous API or the KIFTestActor runBlock: API, yet it is the default for a KIFTestCase.

The -[XCTestCase recordFailureWithDescription:…] method dispatches and raises an _XCTestCaseInterruptionException. The block (calling [_XCTestCaseInterruptionException interruptTest]) will eventually be executed in _dispatch_client_callout which catches the exception and calls objc_terminate. Since this exception is caught inside libdispatch, there’s nothing we can do and the test process will be terminated.

By letting the continueAfterFailure property default to YES it becomes possible to write a test like this which doesn't terminate the test process with an uncaught exception:

- (void) testAsynchronousFailure
{
    dispatch_async(dispatch_get_main_queue(), ^{
        XCTFail(@"This failing assertion causes the test process to terminate because of uncaught exception of type _XCTestCaseInterruptionException if continueAfterFailure == NO");
        [[NSNotificationCenter defaultCenter] postNotificationName:@"AsnychronousTaskDidComplete" object:self];
    });
    [system waitForNotificationName:@"AsnychronousTaskDidComplete" object:self];
}

[phatmann]

The reason continueAfterFailure is set to NO is because KIF tests are a series of steps, and if one step fails then there is no point in continuing with the tests. For example, if we set it to YES and an element is not found, then all the steps that work with that element would also fail, making for a noisy failure report.

Is there any reason you cannot set continueAfterFailure to YES when a test needs asynchronous support? Alternately, we could provide some kind of async test construct that took a block and set continueAfterFailure appropriately.

[0xced]

Of course it doesn’t make sense to continue after a failure in a series of steps. I’ll try to come up with a better solution.