Any way to test that fatalError() is called?

[abbeycode]

I'm testing failure cases in my code, and it doesn't look like Nimble provides a way to assert that a given block will call fatalError(). Is that correct? If so, it seems that would be a useful thing to have.

This Stack Overflow question has a couple of answers that might provide solutions: http://stackoverflow.com/questions/32873212/unit-test-fatalerror-in-swift

[modocache]

I believe it's true that Nimble doesn't provide this functionality. I think it would be very interesting to support this, but I'm not sure it's possible. The Swift stdlib's unit tests have a way to do this; perhaps we can borrow some concepts from there?

[modocache]

@morganchen12 was kind enough to point out that this is being discussed on the Swift mailing lists.

The mailing list discussion is rooted in XCTest, which is an interesting topic indeed. Nimble tries to limit its dependency on XCTest. Perhaps we could pioneer an approach and then submit an evolution proposal for it to be added to XCTest as well?

[abbeycode]

I started looking into how fatalError() is implemented. This is what I found:

swift/stdlib/public/core/Assert.swift:137:

/// Unconditionally print a `message` and stop execution.
@_transparent @noreturn
public func fatalError(
  @autoclosure message: () -> String = String(),
  file: StaticString = __FILE__, line: UInt = __LINE__
) {
  _assertionFailed("fatal error", message(), file, line)
}

_assertionFailed(...) is defined in swift/stdlib/public/core/AssertCommon.swift:115:

/// This function should be used only in the implementation of user-level
/// assertions.
///
/// This function should not be inlined because it is cold and it inlining just
/// bloats code.
@noreturn @inline(never)
@_semantics("stdlib_binary_only")
func _assertionFailed(
  prefix: StaticString, _ message: String,
  _ file: StaticString, _ line: UInt
) {
  // ...
  // Writes out the message in a safe way
  // ...
  Builtin.int_trap()
}

I can't see where int_trap() is defined, though, and how we might be able to intercept it.

[morganchen12]

I would imagine Builtin.int_trap() is just llvm.trap(), which is platform-dependent and just emits an undefined trap instruction or abort() if in doubt. I don't think it can be intercepted in a way that isn't painful.

[abbeycode]

I thought I had something...

[abbeycode]

I've been working on incorporating the solution proposed in a Stack Overflow answer, but need some guidance.

The problem is that it's a two-part solution. The first part overrides fatalError() in client code, in a way that unit tests can swap out the implementation. I don't believe Nimble currently has any component meant to be linked into a client project though, correct? Would a solution that required that be out of bounds?

You would basically need to compile a single Swift file into the client code. I'm not sure how we would approach packaging this type of change for distribution.

[morganchen12]

@abbeycode I don't think that's a perfect solution either--even if we do find a way to inject a file into the code we're trying to test, it'll create duplicate symbols within the same module if they've also tried to add that same file and in either case it won't work if they're calling Swift.fatalError() instead of just fatalError(). Ideally we want our solution to be completely agnostic of the codebase we're testing.

However, there is some good news: http://cocoawithlove.com/blog/2016/02/02/partial-functions-part-two-catching-precondition-failures.html

[jeffh]

Yeah, @morganchen12's proposal is preferred. I don't really like having a test-helper library that is linked in the main app.

Since we can't fork(), mach exception handling is probably the best solution. Otherwise fatalError() may trigger a SIGILL, although I haven't verified that or not.

[abbeycode]

@morganchen12 That's awesome. I'm beginning to implement a solution using Matt's code now.

[abbeycode]

Would it be preferrable to use Matt's code as a submodule, or to copy/appropriate it?

[modocache]

Just a quick note: I haven't had time to think about it in depth, but no matter how we incorporate fatalError testing, we should give attribution to @mattgallagher in some way. It looks like his code is going to have a heavy influence on the code we'll be including in Nimble either way. We should probably also reach out to @mattgallagher and make sure he's OK with whatever approach we end up taking--once we decide what that is.