Return the correct exit code if all tests have been skipped in unittest

[EliseevEgor]

Bug report

In Python 3.12, a new exit code (5, instead of 0) was added if all tests were skipped in unittest. But in some cases it still returns 0. See code examples.

Code example 1:

import unittest


class TestSimple(unittest.TestCase):

    @classmethod
    def setUpClass(cls):
        raise unittest.SkipTest("Skip whole Case")

    def test_true(self):
        self.assertTrue(True)

    def test_false(self):
        self.assertTrue(False, msg="Is not True")

Output:

Ran 0 tests in 0.000s

NO TESTS RAN (skipped=1)

Ok, we get exit code 5 here.

Code example 2:

import unittest


class TestCase(unittest.TestCase):
    @unittest.skip("something")
    def test(self):
        self.assertTrue(False)

Output:

Ran 1 test in 0.000s

OK (skipped=1)

Here we can see output 0. Ran one test and skipped. But I think it is incorrect, because all tests were skipped.

Your environment

• CPython versions tested on: 3.12
• Operating system and architecture: I think it does not matter.

Linked PRs

• gh-106584: Fix exit code for unittest in Python 3.12 #106588
• [3.12] gh-106584: Fix exit code for unittest in Python 3.12 (GH-106588) #109725
• gh-113267: Revert "gh-106584: Fix exit code for unittest in Python 3.12 (#106588)" #114470

[hugovk]

Thank you for the fix!

[serhiy-storchaka]

I think that it was a wrong change, and not only because it makes startTest() and stopTest() calls unbalanced (#113267), but because it does not solve the original issue. The third example should be equivalent to the second example:

import unittest

class TestCase(unittest.TestCase):
    def test(self):
        self.skipTest("something")

But it gets error code 0 while the second example gets error code 5.

It also gives a wrong answer to the problem. #113661 fixes it in the opposite way.

[carljm]

Looks like this change has been reverted, and #113661 PRs are merged also, and that issue is closed. Seems there is nothing left to do here?