Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

interrupted tests apparently lead to success #170

Closed
mrtyler opened this issue Jun 3, 2013 · 1 comment
Closed

interrupted tests apparently lead to success #170

mrtyler opened this issue Jun 3, 2013 · 1 comment
Labels

Comments

@mrtyler
Copy link
Contributor

mrtyler commented Jun 3, 2013

Consider this test case from workflow:

14:33:58 tests.bb_run_test DetermineBuildbotTestCase.test_given_valid_buildbot_in_keys ... Invalid buildbot "t" (normalized to 't')- aborting.
14:33:58 INTERRUPTED in 0.00s
14:33:58 
14:33:58 ========================================================================
14:33:58 FAILURES
14:33:58 None!
14:33:58 
14:33:58 FAILED.  22 tests / 5 cases: 21 passed, 0 failed.  (Total test time 0.03s)

Jenkins think this suite passed because Testify returns code 0, but obviously that is not correct. At minimum, we need to correctly report this as an error.

It may also be worth looking at the status string "FAILED. 22 tests / 5 cases: 21 passed, 0 failed." since it contains several lies.

@asottile
Copy link
Contributor

asottile commented Jan 2, 2015

Still a bug afaict:

$ cat test2.py
import testify as T

class F(T.TestCase):
    def test(self):
        raise KeyboardInterrupt
$ testify test2.py
-
FAILED.  1 test / 1 case: 0 passed, 0 failed.  (Total test time 0.00s)
$ echo $?
0

asottile added a commit that referenced this issue Jan 5, 2015
Exit nonzero when interrupted.  Closes #170
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants