Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[qa] Add --failfast option to functional test runner #13105

Merged
merged 2 commits into from Apr 30, 2018

Conversation

Projects
None yet
6 participants
@jamesob
Copy link
Member

jamesob commented Apr 27, 2018

Add the option (--failfast) to stop the functional test runner's execution when it encounters the first failure.

Also cleans up run_test's arguments list (no more mutable default for args) and call site.

test/functional/test_runner.py Outdated
)

def run_tests(test_list, src_dir, build_dir, tmpdir,
jobs=1, enable_coverage=False, args=None, combined_logs_len=0, failfast=False):

This comment has been minimized.

@MarcoFalke

MarcoFalke Apr 27, 2018

Member

nit: Can keep this in one line, so that it doesn't break when you e.g. grep for run_tests.

@jamesob jamesob force-pushed the jamesob:2018-04-testrunner-failfast branch 3 times, most recently Apr 27, 2018

@jamesob

This comment has been minimized.

Copy link
Member Author

jamesob commented Apr 27, 2018

@MarcoFalke thanks for the look. I've addressed the formatting nit and the process cleanup issue you pointed out.

@theuni

This comment has been minimized.

Copy link
Member

theuni commented Apr 27, 2018

Concept ACK. Any reason not to enable this for Travis?

@jamesob jamesob force-pushed the jamesob:2018-04-testrunner-failfast branch Apr 27, 2018

@jamesob

This comment has been minimized.

Copy link
Member Author

jamesob commented Apr 27, 2018

@theuni good point; I guess that'd encourage a shift in compute consumption onto developers who write buggy code vs. scarce Travis resources. Will add.

jamesob added some commits Apr 27, 2018

Add --failfast option to functional test runner
Also cleans up run_test's arguments list (no more mutable default for `args`)
and call site.

@jamesob jamesob force-pushed the jamesob:2018-04-testrunner-failfast branch to 58f9a0a Apr 27, 2018

@fanquake fanquake added the Tests label Apr 27, 2018

@laanwj

This comment has been minimized.

Copy link
Member

laanwj commented Apr 28, 2018

Concept ACK

@practicalswift

This comment has been minimized.

Copy link
Member

practicalswift commented Apr 28, 2018

Concept ACK

Very nice!

@laanwj

This comment has been minimized.

Copy link
Member

laanwj commented Apr 30, 2018

Tested ACK 58f9a0a
I made the first test that runs here (wallet_hd.py) fail.
Without --failfast it continues after the error.
With --failfast it stopped immediately.

@laanwj laanwj merged commit 58f9a0a into bitcoin:master Apr 30, 2018

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

laanwj added a commit that referenced this pull request Apr 30, 2018

Merge #13105: [qa] Add --failfast option to functional test runner
58f9a0a Use --failfast when running functional tests on Travis (James O'Beirne)
bf720c1 Add --failfast option to functional test runner (James O'Beirne)

Pull request description:

  Add the option (`--failfast`) to stop the functional test runner's execution when it encounters the first failure.

  Also cleans up run_test's arguments list ([no more mutable default for `args`](http://docs.python-guide.org/en/latest/writing/gotchas/#mutable-default-arguments)) and call site.

Tree-SHA512: e854b1b1634bf613ae8ae88e715df1460982fa68db9d785aafeb5eccf5bf324c7f20dded2ca6840ebf18a28347ecac2138d6c7592507b34939b02609ef55e1b3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.