Only use stdlib based unittest runner for py >=3.5 #265

mtreinish · 2019-08-16T14:39:12Z

The stdlib python test runner introduced in #256 was written assuming a
python stdlib from python 3.5 or newer. Since we only officially support
running on python 3.5 or newer when we added the logic to switch between
the stdlib runner and the old testtools based runner we just said
python >=3.0.0. However, despite our official support status for
python >2.7,<3.5 there are still users out there running stestr on these python
versions. The new test runner will not work for those users because
unittest in stdlib changed in python 3.5 (as an aside this is the same
issue I've been hitting in my attempts to get unittest2 removed from
testtools). To address this problem a simple fix is to just change the
if statement to be a bit more explicit and only use the new runner for
python 3.5 or newer. This way stestr is more explicit about where the
newer runner actually works and this should hopefully unbreak any users
out there still using python 3.0 to 3.4.

Fixes #264

The stdlib python test runner introduced in #256 was written assuming a python stdlib from python 3.5 or newer. Since we only officially support running on python 3.5 or newer when we added the logic to switch between the stdlib runner and the old testtools based runner we just said python >=3.0.0. However, despite our official support status for python >2.7,<3.5 there are still users out there running stestr on these python versions. The new test runner will not work for those users because unittest in stdlib changed in python 3.5 (as an aside this is the same issue I've been hitting in my attempts to get unittest2 removed from testtools). To address this problem a simple fix is to just change the if statement to be a bit more explicit and only use the new runner for python 3.5 or newer. This way stestr is more explicit about where the newer runner actually works and this should hopefully unbreak any users out there still using python 3.0 to 3.4. Fixes #264

coveralls · 2019-08-16T14:49:50Z

Coverage remained the same at 68.601% when pulling 8ae34af on only-use-stdlib-runner-py35 into ee4e956 on master.

Held back the following lxml===4.4.1 nova tests fail https://bugs.launchpad.net/nova/+bug/1838666 websockify===0.9.0 tempest test failing, ping nova tornado===5.1.1 salt is cauing this, no eta on fix (same as the last year) stestr===2.5.0 needs merged mtreinish/stestr#265 jsonschema===3.0.2 see https://review.opendev.org/649789 oauthlib===3.1.0 keystone https://bugs.launchpad.net/keystone/+bug/1839393 kubernetes===10.0.1 openshift PINS this, only kuryr-tempest-plugin deps on it tenacity===5.1.1 failing tests in neutron Change-Id: I79baeee85f93adb2adfd01b1f02ca511073c8d78

* Update requirements from branch 'master' - Merge "Updated from generate-constraints" - Updated from generate-constraints Held back the following lxml===4.4.1 nova tests fail https://bugs.launchpad.net/nova/+bug/1838666 websockify===0.9.0 tempest test failing, ping nova tornado===5.1.1 salt is cauing this, no eta on fix (same as the last year) stestr===2.5.0 needs merged mtreinish/stestr#265 jsonschema===3.0.2 see https://review.opendev.org/649789 oauthlib===3.1.0 keystone https://bugs.launchpad.net/keystone/+bug/1839393 kubernetes===10.0.1 openshift PINS this, only kuryr-tempest-plugin deps on it tenacity===5.1.1 failing tests in neutron Change-Id: I79baeee85f93adb2adfd01b1f02ca511073c8d78

mtreinish · 2019-08-27T14:30:13Z

@masayukig did you have any thoughts on this?

masayukig · 2019-08-28T01:28:05Z

stestr/subunit_runner/run.py

@@ -84,7 +84,7 @@ def _list(self, test):

 def main():
    runner = SubunitTestRunner
-    if sys.version_info[0] >= 3:
+    if sys.version_info[0] >= 3 and sys.version_info[1] >= 5:


It might be better to add a comment/note to understand the magic number '5' here. But I think it's not a blocker for merging because we can check it with git log/blame.

masayukig · 2019-08-28T01:30:04Z

I'm sorry for my delayed review. LGTM

stestr which is used by Tempest internally to run the test switch the customize test runner(which use stdlib unittest) for >=py3.5 else testtools.run.- mtreinish/stestr#265 These two test runner are not compatible due to skip exception handling(due to unittest2). testtools.run treat unittestt.SkipTest as error and stdlib unittest treat unittest2.case.SkipTest raised by testtools.TestCase.skipException. testtool issue: testing-cabal/testtools#272 testtool is not so active now a days and fix is also not decided, let's add a workaround in Tempest to make it work for both test runner based on python version same as used by stestr Tempest should work perfectly with stestr<2.5.0 also which is also handled in this workaround. Change-Id: Ie9c013d4d6851d4deef57c1e4c254a9a34374e5a

* Update tempest from branch 'master' - Merge "Add workaround to handle the testtool skip exception issue" - Add workaround to handle the testtool skip exception issue stestr which is used by Tempest internally to run the test switch the customize test runner(which use stdlib unittest) for >=py3.5 else testtools.run.- mtreinish/stestr#265 These two test runner are not compatible due to skip exception handling(due to unittest2). testtools.run treat unittestt.SkipTest as error and stdlib unittest treat unittest2.case.SkipTest raised by testtools.TestCase.skipException. testtool issue: testing-cabal/testtools#272 testtool is not so active now a days and fix is also not decided, let's add a workaround in Tempest to make it work for both test runner based on python version same as used by stestr Tempest should work perfectly with stestr<2.5.0 also which is also handled in this workaround. Change-Id: Ie9c013d4d6851d4deef57c1e4c254a9a34374e5a

stestr which is used by Tempest internally to run the test switch the customize test runner(which use stdlib unittest) for >=py3.5 else testtools.run.- mtreinish/stestr#265 These two test runner are not compatible due to skip exception handling(due to unittest2). testtools.run treat unittestt.SkipTest as error and stdlib unittest treat unittest2.case.SkipTest raised by testtools.TestCase.skipException. testtool issue: testing-cabal/testtools#272 testtool is not so active now a days and fix is also not decided, let's add a workaround in Tempest to make it work for both test runner based on python version same as used by stestr Tempest should work perfectly with stestr<2.5.0 also which is also handled in this workaround. Change-Id: Ie9c013d4d6851d4deef57c1e4c254a9a34374e5a

* Update tempest from branch 'master' to 140d693f8589a15636ddadb68941c7a580eeff75 - Merge "Remove usage of unittest2" - Remove usage of unittest2 from comments when it was last touched it looks like workarounds for unittest2 might be able to be dropped. related: mtreinish/stestr#265 simplify the workaround logic around unittest2 TestCase logic Change-Id: Ibac9d0c2fa2f30605dd44ee58b84946464ea6449

from comments when it was last touched it looks like workarounds for unittest2 might be able to be dropped. related: mtreinish/stestr#265 simplify the workaround logic around unittest2 TestCase logic Change-Id: Ibac9d0c2fa2f30605dd44ee58b84946464ea6449

Held back the following lxml===4.4.1 nova tests fail https://bugs.launchpad.net/nova/+bug/1838666 websockify===0.9.0 tempest test failing, ping nova tornado===5.1.1 salt is cauing this, no eta on fix (same as the last year) stestr===2.5.0 needs merged mtreinish/stestr#265 jsonschema===3.0.2 see https://review.opendev.org/649789 oauthlib===3.1.0 keystone https://bugs.launchpad.net/keystone/+bug/1839393 kubernetes===10.0.1 openshift PINS this, only kuryr-tempest-plugin deps on it Change-Id: I79baeee85f93adb2adfd01b1f02ca511073c8d78

Held back the following lxml===4.4.1 nova tests fail https://bugs.launchpad.net/nova/+bug/1838666 tornado===5.1.1 salt is cauing this, no eta on fix (same as the last year) websockify===0.9.0 tempest test failing, ping nova stestr===2.5.0 needs merged mtreinish/stestr#265 jsonschema===3.0.2 see https://review.opendev.org/649789 oauthlib===3.1.0 keystone https://bugs.launchpad.net/keystone/+bug/1839393 Change-Id: I79baeee85f93adb2adfd01b1f02ca511073c8d78

Held back the following lxml===4.4.1 nova tests fail https://bugs.launchpad.net/nova/+bug/1838666 websockify===0.9.0 tempest test failing, ping nova stestr===2.5.0 needs merged mtreinish/stestr#265 jsonschema===3.0.2 see https://review.opendev.org/649789 oauthlib===3.1.0 keystone https://bugs.launchpad.net/keystone/+bug/1839393 Change-Id: I79baeee85f93adb2adfd01b1f02ca511073c8d78

mtreinish mentioned this pull request Aug 16, 2019

Issue with stestr 2.5.0 using Python 3.4 #264

Closed

mtreinish requested a review from masayukig August 17, 2019 11:55

Merge branch 'master' into only-use-stdlib-runner-py35

8ae34af

masayukig approved these changes Aug 28, 2019

View reviewed changes

masayukig merged commit 403352d into master Aug 28, 2019

masayukig deleted the only-use-stdlib-runner-py35 branch August 28, 2019 01:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Only use stdlib based unittest runner for py >=3.5 #265

Only use stdlib based unittest runner for py >=3.5 #265

mtreinish commented Aug 16, 2019 •

edited

Loading

coveralls commented Aug 16, 2019 •

edited

Loading

mtreinish commented Aug 27, 2019

masayukig Aug 28, 2019

masayukig commented Aug 28, 2019

Only use stdlib based unittest runner for py >=3.5 #265

Only use stdlib based unittest runner for py >=3.5 #265

Conversation

mtreinish commented Aug 16, 2019 • edited Loading

coveralls commented Aug 16, 2019 • edited Loading

mtreinish commented Aug 27, 2019

masayukig Aug 28, 2019

Choose a reason for hiding this comment

masayukig commented Aug 28, 2019

mtreinish commented Aug 16, 2019 •

edited

Loading

coveralls commented Aug 16, 2019 •

edited

Loading