Logging output when using fatal assertions

[jwillikers]

Example

A fatal assertion with logged output for failure.

expect((true == false) >> fatal) << "Incorrect!";

Expected Behavior

When the assertion fails, "Incorrect!" should be printed.

FAILED [false] Incorrect!

Actual Behavior

When the assertion fails, "Incorrect!" is not printed.

FAILED [false]

Steps to Reproduce the Problem

1. A failing fatal assertion with streamed output will fail to print the output.

Specifications

• Version: v1.1.8
• Platform: macOS 10.15 Catalina
• Subsystem: MacPorts, LLVM Clang 10.0.1

[kris-jusiak]

Sorry for the late response. I wasn't sure what should be done about it but after some thinking, I get into the conclusion that we don't wanna print in all cases and hence it's better not to print it all not to confuse the users 🤔 But I'm happy to be
convinced otherwise ☝️

To illustrate what I'm talking about let's magine the case that after the fatal assertion we will try to print out range value - that would sig segv 😞

std::vector v{1, 2, 3};
(std::size(v) == 3_i >> fatal) << "incorrect" << v[4];

[jwillikers]

Wouldn't that be an issue even if the assertion was not fatal or am I missing something?

[kris-jusiak]

Yes, it would but that's why you would use a fatal assertion to avoid it, right?

[jwillikers]

Aww, I see what you mean. What if a user wants helpful error information when a fatal assertion occurs? This is my use case.

[kris-jusiak]

I understand the use case (which is obviously a valid one) just have concerns that by allowing streaming anything after fatal assertion the guarantee that the assertion will bail out and won't crash (the main purpose of it) may be broken. It's not an issue if you just print a string or something but it might be if a value out of range will be tried to be accessed. We potentially can allow just safe output in the output stream after fatal so that the sig segv ain't gonna happen, not sure 🤔

[jwillikers]

This appears to come down to a ease-of-use / safety problem. A user could just workaround this situation by using an if statement to log output and then explicitly fail the test but this isn't very convenient. I like your idea about safe output - I'm guessing safe output would need to evaluate to a string at compile-time. Perhaps an opt-in approach could allow the unsafe behavior, i.e.:

1. Safe, compiles, and logs the given output as expected.

std::vector v{1, 2, 3};
(std::size(v) == 2_i >> fatal) << "Surprised?";

2. Unsafe so it fails to compile.

std::vector v{1, 2, 3};
(std::size(v) == 3_i >> fatal) << "I like to play it safe." << v[4];

3. Unsafe but compiles because the user explicitly opted in.

std::vector v{1, 2, 3};
(std::size(v) == 3_i >> fatal) << "I swear I know what I'm doing!" << unsafe << v[4];

[garymm]

I don't understand the concern. The message will only be printed in the case that the assertion fails, right? So using the assertion to guard against some unsafe behavior in the message seems very strange.
In any case this is very counter to all other testing frameworks I've used and caused me a lot of wasted time trying to figure out where my test was failing.