-
Notifications
You must be signed in to change notification settings - Fork 35.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
qa: Warn when specified test is not found #10374
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
utACK fa57826bbf14e16a959b945a1cf74d2c58f257df
Everything looks good other than a couple of nits. Will test once nits addressed.
test/functional/test_runner.py
Outdated
@@ -173,7 +173,7 @@ def main(): | |||
args, unknown_args = parser.parse_known_args() | |||
|
|||
# Create a set to store arguments and create the passon string | |||
tests = set(arg for arg in unknown_args if arg[:2] != "--") | |||
tests = [arg for arg in unknown_args if arg[:2] != "--"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is great functionality to allow tests to be run multiple times. It might be useful to edit https://github.com/MarcoFalke/bitcoin/blob/Mf1705-qaWarnTestNotFound/test/README.md#functional-tests to specify that you can, in fact, run the same test a few times to catch intermittent errors.
tests_excl = [re.sub("\.py$", "", t) + ".py" for t in args.exclude.split(',')] | ||
for exclude_test in tests_excl: | ||
if exclude_test in test_list: | ||
test_list.remove(exclude_test) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This will only remove the first instance of exclude_test
from test_list
. It's now possible that there are multiple instances of the script from line 176 above. Something like this will work:
test_list = list(filter(lambda t: t == exclude_test, test_list))
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a feature. If you really want to specify the same test multiple times, and then exclude it again, you need to specify it multiple times as well. Though, I doubt this will ever be relevant.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tested ACK fa57826bbf14e16a959b945a1cf74d2c58f257df
test/functional/test_runner.py
Outdated
@@ -173,7 +173,7 @@ def main(): | |||
args, unknown_args = parser.parse_known_args() | |||
|
|||
# Create a set to store arguments and create the passon string |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: comment is now incorrect (since this is a list rather than a set)
fa57826
to
fac79e4
Compare
Thx for the feedback. Updated doc with --amend |
utACK fac79e4 |
fac79e4 qa: Warn when specified test is not found (MarcoFalke) Tree-SHA512: d11ecdde275309b12e23155f6cd8e26c99217436b5094a70dd51b95ae7688754227628dd9a801eb6a52ff3ebea4420938e2fc8e9dc9cd77a4dd5c28d2b822354
fac79e4 qa: Warn when specified test is not found (MarcoFalke) Tree-SHA512: d11ecdde275309b12e23155f6cd8e26c99217436b5094a70dd51b95ae7688754227628dd9a801eb6a52ff3ebea4420938e2fc8e9dc9cd77a4dd5c28d2b822354
fac79e4 qa: Warn when specified test is not found (MarcoFalke) Tree-SHA512: d11ecdde275309b12e23155f6cd8e26c99217436b5094a70dd51b95ae7688754227628dd9a801eb6a52ff3ebea4420938e2fc8e9dc9cd77a4dd5c28d2b822354
fac79e4 qa: Warn when specified test is not found (MarcoFalke) Tree-SHA512: d11ecdde275309b12e23155f6cd8e26c99217436b5094a70dd51b95ae7688754227628dd9a801eb6a52ff3ebea4420938e2fc8e9dc9cd77a4dd5c28d2b822354
fac79e4 qa: Warn when specified test is not found (MarcoFalke) Tree-SHA512: d11ecdde275309b12e23155f6cd8e26c99217436b5094a70dd51b95ae7688754227628dd9a801eb6a52ff3ebea4420938e2fc8e9dc9cd77a4dd5c28d2b822354
fac79e4 qa: Warn when specified test is not found (MarcoFalke) Tree-SHA512: d11ecdde275309b12e23155f6cd8e26c99217436b5094a70dd51b95ae7688754227628dd9a801eb6a52ff3ebea4420938e2fc8e9dc9cd77a4dd5c28d2b822354
fac79e4 qa: Warn when specified test is not found (MarcoFalke) Tree-SHA512: d11ecdde275309b12e23155f6cd8e26c99217436b5094a70dd51b95ae7688754227628dd9a801eb6a52ff3ebea4420938e2fc8e9dc9cd77a4dd5c28d2b822354
fac79e4 qa: Warn when specified test is not found (MarcoFalke) Tree-SHA512: d11ecdde275309b12e23155f6cd8e26c99217436b5094a70dd51b95ae7688754227628dd9a801eb6a52ff3ebea4420938e2fc8e9dc9cd77a4dd5c28d2b822354
fac79e4 qa: Warn when specified test is not found (MarcoFalke) Tree-SHA512: d11ecdde275309b12e23155f6cd8e26c99217436b5094a70dd51b95ae7688754227628dd9a801eb6a52ff3ebea4420938e2fc8e9dc9cd77a4dd5c28d2b822354
fac79e4 qa: Warn when specified test is not found (MarcoFalke) Tree-SHA512: d11ecdde275309b12e23155f6cd8e26c99217436b5094a70dd51b95ae7688754227628dd9a801eb6a52ff3ebea4420938e2fc8e9dc9cd77a4dd5c28d2b822354
fac79e4 qa: Warn when specified test is not found (MarcoFalke) Tree-SHA512: d11ecdde275309b12e23155f6cd8e26c99217436b5094a70dd51b95ae7688754227628dd9a801eb6a52ff3ebea4420938e2fc8e9dc9cd77a4dd5c28d2b822354
fac79e4 qa: Warn when specified test is not found (MarcoFalke) Tree-SHA512: d11ecdde275309b12e23155f6cd8e26c99217436b5094a70dd51b95ae7688754227628dd9a801eb6a52ff3ebea4420938e2fc8e9dc9cd77a4dd5c28d2b822354
fac79e4 qa: Warn when specified test is not found (MarcoFalke) Tree-SHA512: d11ecdde275309b12e23155f6cd8e26c99217436b5094a70dd51b95ae7688754227628dd9a801eb6a52ff3ebea4420938e2fc8e9dc9cd77a4dd5c28d2b822354
fac79e4 qa: Warn when specified test is not found (MarcoFalke) Tree-SHA512: d11ecdde275309b12e23155f6cd8e26c99217436b5094a70dd51b95ae7688754227628dd9a801eb6a52ff3ebea4420938e2fc8e9dc9cd77a4dd5c28d2b822354
fac79e4 qa: Warn when specified test is not found (MarcoFalke) Tree-SHA512: d11ecdde275309b12e23155f6cd8e26c99217436b5094a70dd51b95ae7688754227628dd9a801eb6a52ff3ebea4420938e2fc8e9dc9cd77a4dd5c28d2b822354
fac79e4 qa: Warn when specified test is not found (MarcoFalke) Tree-SHA512: d11ecdde275309b12e23155f6cd8e26c99217436b5094a70dd51b95ae7688754227628dd9a801eb6a52ff3ebea4420938e2fc8e9dc9cd77a4dd5c28d2b822354
fac79e4 qa: Warn when specified test is not found (MarcoFalke) Tree-SHA512: d11ecdde275309b12e23155f6cd8e26c99217436b5094a70dd51b95ae7688754227628dd9a801eb6a52ff3ebea4420938e2fc8e9dc9cd77a4dd5c28d2b822354
fac79e4 qa: Warn when specified test is not found (MarcoFalke) Tree-SHA512: d11ecdde275309b12e23155f6cd8e26c99217436b5094a70dd51b95ae7688754227628dd9a801eb6a52ff3ebea4420938e2fc8e9dc9cd77a4dd5c28d2b822354
This will warn on typos on tests provided manually on the command line.
Also, this comes with the new feature that the same test can be specified multiple times on the command line.