New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
tests(fix): lmtp_ip.bats
improve partial failure output
#3552
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Instead of exit status of `124` (_signifies timeout_), it should fail with `1` (failure) like the others. Handled via using `_run_in_container_bash()` (_`timeout` failure `124` does not propagate, and is treated as `1` instead_). In this case we are waiting on the status of the mail being sent, the pattern provided to `grep` is too specific and results in a timeout. Instead since we only expect the one log entry, match any status and assert the expected pattern afterwards. This provides a more helpful failure output that informs us that mail was at least processed by Postfix, but the sent status is not what we expected. ### Before ``` ✗ [ENV] (POSTFIX_DAGENT) delivers mail to existing account [60327] (from function `assert_success' in file test/test_helper/bats-assert/src/assert_success.bash, line 42, in test file test/tests/parallel/set3/mta/lmtp_ip.bats, line 47) `assert_success' failed -- command failed -- status : 124 output : -- ``` ### After ``` ✗ [ENV] (POSTFIX_DAGENT) delivers mail to existing account [1425] (from function `assert_output' in file test/test_helper/bats-assert/src/assert_output.bash, line 178, in test file test/tests/parallel/set3/mta/lmtp_ip.bats, line 48) `assert_output --regexp "${MATCH_LOG_LINE}=sent .* Saved)"' failed -- regular expression does not match output -- regexp : postfix/lmtp.* status=sent .* Saved) output : Sep 28 04:12:52 mail postfix/lmtp[721]: 23701B575: to=<user1@localhost.localdomain>, relay=127.0.0.1[127.0.0.1]:24, delay=0.08, delays=0.07/0/0.01/0, dsn=4.2.0, status=deferred (host 127.0.0.1[127.0.0.1] said: 451 4.2.0 <user1@localhost.localdomain> Internal error occurred. Refer to server log for more information. [2023-09-28 04:12:52] (in reply to end of DATA command)) -- ``` The expected pattern is logged as `assert_success` confirms a valid match for the log line of interest was found, and we have the mismatched value to debug the failure against.
polarathene
added
service/postfix
area/tests
kind/improvement
Improve an existing feature, configuration file or the documentation
labels
Sep 28, 2023
For reference, this approach with a timeout and Other tests that run a command repeatedly until timeout via helper can get lengthy depending on output:
|
This was referenced Sep 28, 2023
georglauterbach
approved these changes
Sep 28, 2023
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍🏼
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
area/tests
kind/improvement
Improve an existing feature, configuration file or the documentation
service/postfix
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Instead of exit status of
124
(signifies timeout), it should fail with1
(failure) like the others. Handled via using_run_in_container_bash()
(timeout
failure124
does not propagate, and is treated as1
instead).In this case we are waiting on the status of the mail being sent, the pattern provided to
grep
is too specific and results in a timeout. Instead since we only expect the one log entry, match any status and assert the expected pattern afterwards.This provides a more helpful failure output that informs us that mail was at least processed by Postfix, but the sent status is not what we expected.
Before
After
The expected pattern is logged as
assert_success
confirms a valid match for the log line of interest was found, and we have the mismatched value to debug the failure against.Type of change
Checklist: