I find then when writing custom assertions, the output is often not what I expect:
Custom assertion failures print assertion name instead of test name
If I have:
assert_valid_json() {
# If $1 was not valid json ...
fail "Invalid json: $1"
}
test_check_response() {
local response="$(...)"
assert_valid_json "$response"
}I expect to see:
✗ Failed: Check response
Message: Invalid json: ...But instead, it uses the name of the function where fail (or any other bashunit assertion) produced the failure:
✗ Failed: Assert valid json
Message: Invalid json: ...This means that I need to be very careful to only use bashunit assertions in test_... methods, and I can't factor out helper methods and use those from my functions.
Perhaps I'm doing something wrong. 🙂 I've worked around this by writing a function that walks up the call stack until it encounters the first method whose name starts with 'test_' and uses that as the test name via state::print_line.
Custom Assertions Print Failures Twice
When I use 'console_results::print_failure_message', I will get the failure message twice in the non-summary output, e.g.:
(I wrote a test that should fail here, using a custom assertion similar to assert_exit_code but with clearer output)
Running tests.sh
✗ Failed: Get foo should fail
Command failed: get-foo, got exit code 0
✗ Failed: Get foo should fail <-- Why printed twice?
Command failed: get-foo, got exit code 0
There was 1 failure:
|1) tests.sh:246
|✗ Failed: Get foo should fail
| Command failed: get-foo, got exit code 0
This does not happen if I use fail. I am only using console_results::print_failure_message because of the need to specify the test function name, per the previous item.
Using Assertions in a Custom Assertion Will Always Run All of Them, Even if One Fails
If I write assert_foo() and, inside it, I try to assert_exit_code and assert_file_contains, it will always run both assertions, even if the first fails, leading to strange errors.
I find then when writing custom assertions, the output is often not what I expect:
Custom assertion failures print assertion name instead of test name
If I have:
I expect to see:
✗ Failed: Check response Message: Invalid json: ...But instead, it uses the name of the function where
fail(or any other bashunit assertion) produced the failure:✗ Failed: Assert valid json Message: Invalid json: ...This means that I need to be very careful to only use bashunit assertions in
test_...methods, and I can't factor out helper methods and use those from my functions.Perhaps I'm doing something wrong. 🙂 I've worked around this by writing a function that walks up the call stack until it encounters the first method whose name starts with 'test_' and uses that as the test name via
state::print_line.Custom Assertions Print Failures Twice
When I use 'console_results::print_failure_message', I will get the failure message twice in the non-summary output, e.g.:
(I wrote a test that should fail here, using a custom assertion similar to assert_exit_code but with clearer output)
Running tests.sh ✗ Failed: Get foo should fail Command failed: get-foo, got exit code 0 ✗ Failed: Get foo should fail <-- Why printed twice? Command failed: get-foo, got exit code 0 There was 1 failure: |1) tests.sh:246 |✗ Failed: Get foo should fail | Command failed: get-foo, got exit code 0This does not happen if I use
fail. I am only usingconsole_results::print_failure_messagebecause of the need to specify the test function name, per the previous item.Using Assertions in a Custom Assertion Will Always Run All of Them, Even if One Fails
If I write
assert_foo()and, inside it, I try toassert_exit_codeandassert_file_contains, it will always run both assertions, even if the first fails, leading to strange errors.