Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fail if baseline execution fails #181

Merged
merged 3 commits into from Nov 8, 2016

Conversation

@atodorov
Copy link
Contributor

commented Nov 3, 2016

@abingham this is ready for review.

Please note the first commit in this PR. For some reason I was missing the run_tests.sh plumbing from #160 so I did cherry-pick of 2718e81 and adjusted for latest syntax. If you can figure out what went wrong I will be interested to hear, especially if you manage to find steps to reproduce. The only thing that comes to mind is accidentally removing this piece of code while dealing with merge conflicts on Jul 24, 2016.

atodorov added some commits Jul 20, 2016

Add tests for empty __init__.py
this is the missing part from commit 2718e81 updated to latest
syntax!
Measure baseline timeout in cli.py
this avoids the use of subprocess.call() and makes it easier to
return errors in case baseline fails b/c we don't have to do
cyclic imports.

@atodorov atodorov force-pushed the MrSenko:fail_baseline branch from d70763f to ce719e2 Nov 3, 2016

Fail if baseline fails. Fixes #111 and #169
Cosmic Ray exits with status 2 if baseline test execution fails.
Also prints on the console anything that was returned by the test
runner.

@atodorov atodorov force-pushed the MrSenko:fail_baseline branch from ce719e2 to db7b7c6 Nov 3, 2016

@atodorov

This comment has been minimized.

Copy link
Contributor Author

commented Nov 8, 2016

@abingham ping

configuration['<top-module>'],
configuration['<test-args>'])
with Timer() as t:
handle_baseline(configuration)

This comment has been minimized.

Copy link
@abingham

abingham Nov 8, 2016

Contributor

This is really nice!

@@ -39,18 +37,3 @@ def __enter__(self):

def __exit__(self, ex_type, ex_value, ex_traceback):
pass


def run_baseline(test_runner, module_name, test_args):

This comment has been minimized.

Copy link
@abingham

abingham Nov 8, 2016

Contributor

I'm a little bit concerned since the baseline is now timing one thing - the time to run the tests - while the worker is timing something else - the time to start a python process and run the tests. This is probably not a practical issue for most cases, but it's worth keeping in mind.

@abingham

This comment has been minimized.

Copy link
Contributor

commented Nov 8, 2016

Sorry for forgetting about this!

@abingham abingham merged commit 6fc6fb5 into sixty-north:master Nov 8, 2016

1 check failed

continuous-integration/travis-ci/pr The Travis CI build failed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.