test: Make summary row bold-red if any test failed and show failed tests at end of table #12811
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Make the summary row of the test runner bold red if any test fails. This helps visibility if something fails.
|
Added a commit (by @ajtowns) that moves the failed tests to the bottom of the list, to avoid needing to scroll to find out what failed. |
laanwj
changed the title
laanwj
changed the title
|
utACK d2c6a37 |
jnewbery
reviewed
Mar 28, 2018
test/functional/test_runner.py
Outdated
| elif self.status == "Failed": | ||
| return 2, self.name.lower() | ||
| elif self.status == "Skipped": | ||
| return 0, self.name.lower() |
There was a problem hiding this comment.
My preference would be to place skipped tests below passed tests for visibility, so the order would be:
- passed tests
- skipped tests
- failed tests
- SUMMARY
There was a problem hiding this comment.
Fair point, I'll swap the passed and skipped
Change sorting output to put failed tests at the end of test_runner output.
laanwj
force-pushed
the
2018_03_tests_summaryrow
branch
from
d2c6a37
to
ffb033a
Compare
|
Tested ACK ffb033a |
laanwj
added a commit
that referenced
this pull request
Mar 28, 2018
…show failed tests at end of table ffb033a test: List any failed tests at the end of test_runner output (Anthony Towns) f92541f test: Make summary row bold-red if any test failed (Wladimir J. van der Laan) Pull request description: Make the summary row of the test runner bold red if *any* test fails. This helps visibility if something fails. (yesteryday I had a snafu where I missed that `feature_blocksdir.py` had failed because it's one of the earlier tests in the list, this intends to avoid that in the future) Before:  After:  If tests pass it still looks the same:  Tree-SHA512: 057748c693ca1c80840e4e4cdea8aa1baf8996f03d6805975d8e3c07c4ba0087cd8fa83f891d6bf1af0bfbba88b5d46bd5d852df340d755202bd32ae6f1034b5
Fabcien
pushed a commit
to Bitcoin-ABC/bitcoin-abc
that referenced
this pull request
Aug 30, 2019
…show failed tests at end of table Summary: ffb033a test: List any failed tests at the end of test_runner output (Anthony Towns) f92541f test: Make summary row bold-red if any test failed (Wladimir J. van der Laan) Pull request description: Make the summary row of the test runner bold red if *any* test fails. This helps visibility if something fails. (yesteryday I had a snafu where I missed that `feature_blocksdir.py` had failed because it's one of the earlier tests in the list, this intends to avoid that in the future) Before:  After:  If tests pass it still looks the same:  Tree-SHA512: 057748c693ca1c80840e4e4cdea8aa1baf8996f03d6805975d8e3c07c4ba0087cd8fa83f891d6bf1af0bfbba88b5d46bd5d852df340d755202bd32ae6f1034b5 Backport of Core PR12811 bitcoin/bitcoin#12811 Test Plan: Change a test fail to force failure test_runner.py Reviewers: deadalnix, Fabien, jasonbcox, O1 Bitcoin ABC, #bitcoin_abc, markblundeberg Reviewed By: O1 Bitcoin ABC, #bitcoin_abc, markblundeberg Subscribers: markblundeberg Differential Revision: https://reviews.bitcoinabc.org/D3901
PastaPastaPasta
pushed a commit
to PastaPastaPasta/dash
that referenced
this pull request
Sep 27, 2020
…ed and show failed tests at end of table ffb033a test: List any failed tests at the end of test_runner output (Anthony Towns) f92541f test: Make summary row bold-red if any test failed (Wladimir J. van der Laan) Pull request description: Make the summary row of the test runner bold red if *any* test fails. This helps visibility if something fails. (yesteryday I had a snafu where I missed that `feature_blocksdir.py` had failed because it's one of the earlier tests in the list, this intends to avoid that in the future) Before:  After:  If tests pass it still looks the same:  Tree-SHA512: 057748c693ca1c80840e4e4cdea8aa1baf8996f03d6805975d8e3c07c4ba0087cd8fa83f891d6bf1af0bfbba88b5d46bd5d852df340d755202bd32ae6f1034b5
PastaPastaPasta
pushed a commit
to PastaPastaPasta/dash
that referenced
this pull request
Sep 27, 2020
…ed and show failed tests at end of table ffb033a test: List any failed tests at the end of test_runner output (Anthony Towns) f92541f test: Make summary row bold-red if any test failed (Wladimir J. van der Laan) Pull request description: Make the summary row of the test runner bold red if *any* test fails. This helps visibility if something fails. (yesteryday I had a snafu where I missed that `feature_blocksdir.py` had failed because it's one of the earlier tests in the list, this intends to avoid that in the future) Before:  After:  If tests pass it still looks the same:  Tree-SHA512: 057748c693ca1c80840e4e4cdea8aa1baf8996f03d6805975d8e3c07c4ba0087cd8fa83f891d6bf1af0bfbba88b5d46bd5d852df340d755202bd32ae6f1034b5
PastaPastaPasta
pushed a commit
to PastaPastaPasta/dash
that referenced
this pull request
Oct 14, 2020
…ed and show failed tests at end of table ffb033a test: List any failed tests at the end of test_runner output (Anthony Towns) f92541f test: Make summary row bold-red if any test failed (Wladimir J. van der Laan) Pull request description: Make the summary row of the test runner bold red if *any* test fails. This helps visibility if something fails. (yesteryday I had a snafu where I missed that `feature_blocksdir.py` had failed because it's one of the earlier tests in the list, this intends to avoid that in the future) Before:  After:  If tests pass it still looks the same:  Tree-SHA512: 057748c693ca1c80840e4e4cdea8aa1baf8996f03d6805975d8e3c07c4ba0087cd8fa83f891d6bf1af0bfbba88b5d46bd5d852df340d755202bd32ae6f1034b5
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Make the summary row of the test runner bold red if any test fails. This helps visibility if something fails.
(yesteryday I had a snafu where I missed that
feature_blocksdir.pyhad failed because it's one of the earlier tests in the list, this intends to avoid that in the future)Before:
After:
If tests pass it still looks the same: