-
Notifications
You must be signed in to change notification settings - Fork 176
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
tests should fail if before etc call back with an error or crash #103
Comments
Shouldn't there be a test which is dependent on the results from |
I agree with @garthk on this one, I've been very surprised it acted this way as well. |
|
I'd shamelessly inspire from mocha which seems to consider this as a test which won't appear in the reporting unless it fails (tests are named |
This is partially fixed in 4.0.0. If before/beforeEach fail, the tests are not run. If after/afterEach fails, it is recorded and will cause the final code to be 1 (error). What is missing is to make the skipped tests (due to before failing) explicitly fail in the report. |
Also, in 4.0 we no longer run before/after inside a domain which means if those fail by throwing, the whole thing just stops. We now consider anything outside of the tests to be application code which is the authors responsibility to manage. The before/after callbacks do take an error argument now used to indicate a problem. |
This thread has been automatically locked due to inactivity. Please open a new issue for related bugs or questions following the new issue template instructions. |
If
before
,beforeEach
,after
, orafterEach
call back with an error, the test should fail.Consider the following experiment:
I'd expect that to log "Error: should fail", fail the test without running it, and exit with a non-zero code.
Under
lab@3.2.1
, though:lab
logs the error (good), runs the test anyway (bad), and exits cleanly (bad).Similarly:
… should fail tests, but doesn't.
The text was updated successfully, but these errors were encountered: