New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fail if baseline execution fails #181

Merged
merged 3 commits into from Nov 8, 2016

Conversation

Projects
None yet
2 participants
@atodorov
Contributor

atodorov commented Nov 3, 2016

@abingham this is ready for review.

Please note the first commit in this PR. For some reason I was missing the run_tests.sh plumbing from #160 so I did cherry-pick of 2718e81 and adjusted for latest syntax. If you can figure out what went wrong I will be interested to hear, especially if you manage to find steps to reproduce. The only thing that comes to mind is accidentally removing this piece of code while dealing with merge conflicts on Jul 24, 2016.

atodorov added some commits Jul 20, 2016

Add tests for empty __init__.py
this is the missing part from commit 2718e81 updated to latest
syntax!
Measure baseline timeout in cli.py
this avoids the use of subprocess.call() and makes it easier to
return errors in case baseline fails b/c we don't have to do
cyclic imports.
Fail if baseline fails. Fixes #111 and #169
Cosmic Ray exits with status 2 if baseline test execution fails.
Also prints on the console anything that was returned by the test
runner.
@atodorov

This comment has been minimized.

Show comment
Hide comment
@atodorov

atodorov Nov 8, 2016

Contributor

@abingham ping

Contributor

atodorov commented Nov 8, 2016

@abingham ping

configuration['<top-module>'],
configuration['<test-args>'])
with Timer() as t:
handle_baseline(configuration)

This comment has been minimized.

@abingham

abingham Nov 8, 2016

Contributor

This is really nice!

@abingham

abingham Nov 8, 2016

Contributor

This is really nice!

@@ -39,18 +37,3 @@ def __enter__(self):
def __exit__(self, ex_type, ex_value, ex_traceback):
pass
def run_baseline(test_runner, module_name, test_args):

This comment has been minimized.

@abingham

abingham Nov 8, 2016

Contributor

I'm a little bit concerned since the baseline is now timing one thing - the time to run the tests - while the worker is timing something else - the time to start a python process and run the tests. This is probably not a practical issue for most cases, but it's worth keeping in mind.

@abingham

abingham Nov 8, 2016

Contributor

I'm a little bit concerned since the baseline is now timing one thing - the time to run the tests - while the worker is timing something else - the time to start a python process and run the tests. This is probably not a practical issue for most cases, but it's worth keeping in mind.

@abingham

This comment has been minimized.

Show comment
Hide comment
@abingham

abingham Nov 8, 2016

Contributor

Sorry for forgetting about this!

Contributor

abingham commented Nov 8, 2016

Sorry for forgetting about this!

@abingham abingham merged commit 6fc6fb5 into sixty-north:master Nov 8, 2016

1 check failed

continuous-integration/travis-ci/pr The Travis CI build failed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment