Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests with Result could have nicer output #52436

Closed
steveklabnik opened this issue Jul 16, 2018 · 2 comments
Closed

tests with Result could have nicer output #52436

steveklabnik opened this issue Jul 16, 2018 · 2 comments
Labels
A-diagnostics Area: Messages for errors, warnings, and lints A-libtest Area: #[test] related

Comments

@steveklabnik
Copy link
Member

Consider this test:

#[cfg(test)]
mod tests {
    #[test]
    fn it_works() -> Result<(), String> {
        if 2 + 2 != 4 {
            Ok(())
        } else {
            Err(String::from("two plus two does not equal four"))
        }
    }
}

(note the !) when this fails, you get

> cargo test
   Compiling termination v0.1.0 (file:///C:/Users/steve/tmp/termination)
    Finished dev [unoptimized + debuginfo] target(s) in 1.44s
     Running target\debug\deps\termination-4b340b40460d3098.exe

running 1 test
test tests::it_works ... FAILED

failures:

---- tests::it_works stdout ----
Error: "two plus two does not equal four"
thread 'tests::it_works' panicked at 'assertion failed: `(left == right)`
  left: `1`,
 right: `0`', libtest\lib.rs:326:5

This is because it triggers this assert.

can we make this better somehow? 1 != 0 is not super helpful at learning why this test failed.

@steveklabnik steveklabnik added the A-diagnostics Area: Messages for errors, warnings, and lints label Jul 16, 2018
@estebank estebank added the A-libtest Area: #[test] related label Jul 16, 2018
@scottmcm
Copy link
Member

Hmm, the trait is super-unstable, so maybe it's be worth adding a purpose-built method for this? Like

fn assert_success(self) {
    assert_eq!(self.report(), 0);
}

But then the Result impl could override it to show something better...

@srijs
Copy link
Contributor

srijs commented Jul 17, 2018

What if it printed something like this?

---- tests::it_works stdout ----
The test function returned a termination value that indicates a failure. The detailed report follows.
Error: "two plus two does not equal four"
thread 'tests::it_works' panicked at 'non-successful termination value', libtest\lib.rs:326:5

Or simply

---- tests::it_works stdout ----
Error: "two plus two does not equal four"
thread 'tests::it_works' panicked at 'test function returned termination value that indicates a failure', libtest\lib.rs:326:5

EDIT: Nevermind the first suggestion, that seems like it couldn't work because Termination::report sort of couples printing the message with returning a code.

GuillaumeGomez added a commit to GuillaumeGomez/rust that referenced this issue Aug 15, 2018
improve diagnostics for tests with custom return values

This is an attempt at getting the ball rolling to improve the diagnostics for test functions that return custom `impl Termination` values (see rust-lang#52436).

An alternative could be to use `eprintln!`, but including this in the panic message felt nicely consistent with how failing test assertions would be reported.

Let me know what you think!
@bors bors closed this as completed in 6411aef Aug 15, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-diagnostics Area: Messages for errors, warnings, and lints A-libtest Area: #[test] related
Projects
None yet
Development

No branches or pull requests

4 participants