Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve the failure messages when tests fail. #298

Merged
merged 1 commit into from Jun 30, 2019

Conversation

@Dretch
Copy link
Collaborator

commented Jun 30, 2019

  • The line numbers in the error messages now refer to the function that calls shouldBe*, instead of the location of the assertion inside of shouldBe*.

  • The error messages now include the actual exit code, stdout and stderr from the failed invocation.

  • The slight downside of this change is that the expected value for code/stdout/stderr no longer appears in the error message. Expected values are normally easily visible in the test code itself, so this should not be too painful.

Description of the change

Output from a failed test before this change:

test/Utils.hs:42:3:
  1) PscPackage, spago psc-package-local-setup, Local setup should succeed on first run
       expected: ExitSuccess
        but got: ExitFailure 1

Output after this change:

test/PscPackageSpec.hs:27:45: 
  1) PscPackage, spago psc-package-local-setup, Local setup should succeed on first run
       predicate failed on: (ExitFailure 1,"","spago: spago is broke\n")
Improve the failure messages when tests fail.
- The line numbers in the error messages now refer to the function that
  calls `shouldBe*`, instead of the location of the assertion inside of
  `shouldBe*`.

- The error messages now include the actual exit code, stdout and
  stderr from the failed invocation.

- The slight downside of this change is that the expected value for
  code/stdout/stderr no longer appears in the error message. Expected
  values are normally easily visible in the test code itself, so this
  should not be too painful.
@f-f

f-f approved these changes Jun 30, 2019

Copy link
Member

left a comment

Looks great! 👏

@Dretch Dretch merged commit 7bf293a into master Jun 30, 2019

2 checks passed

Travis CI - Pull Request Build Passed
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details

@Dretch Dretch deleted the better-test-failure-messages branch Jun 30, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.