-
Notifications
You must be signed in to change notification settings - Fork 88
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove unittest2 usage from testtools #277
Conversation
Currently testtools bases all of it's unittest extensions off of unittest2 instead of the stdlib unittest. At one point this made sense since unittest2 provided a stable base as unittest in stdlib added features. But it's been ~5 years since there was a unittest2 release (or a patch merged) and things have changed since then. The best example of this is of the supported python versions listed in the unittest2 project description/README only one is still supported by upstream python, 2.7, which goes end of life at the end of this year. More specific to testtools the use of unittest2 causes a whole slew of issues because of differences in behavior with stdlib unittest. For example here a couple issues encountered: https://bugs.launchpad.net/testtools/+bug/1467558 https://bugs.launchpad.net/testtools/+bug/1417803 mtreinish/stestr#238 testing-cabal#272 which are caused, at least in part, by unittest2. There are likely other bugs related to it that haven't been reported (or I just missed/forgot about). At this point it's better to remove the unittest2 usage and just rely on the upstream stdlib unittest which if nothing else is actively maintained. It'll improve compatibility using the testtools runner with stdlib unittest test suites and removes the class of bugs caused by the differences in unittest2. Fixes testing-cabal#263
+1
|
Would be great to get this in. I ran into this over the last couple days, and took me quite awhile to track it down to the same conclusion and find this issue and PR. |
It's also needed for python 2.7 support, unittest2 was a rolling backport and included some of the test runner (mainly the loader) from stdlib added after 2.7. (I hit a similar issue in: mtreinish/stestr#256 ). This is why the py2.7 CI jobs are failing here. I've opened up #286 to discuss dropping 2.7 support. Once we stop supporting 2.7 in testtools this will be unblocked. |
assertItemsEqual was removed from Python's unittest.TestCase in Python 3.3 [1][2]. We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Change-Id: I095a8aac336e5aa0fbfed64406e33b3b40ff42bc
* Update openstacksdk from branch 'master' - Replace assertItemsEqual with assertCountEqual assertItemsEqual was removed from Python's unittest.TestCase in Python 3.3 [1][2]. We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Change-Id: I095a8aac336e5aa0fbfed64406e33b3b40ff42bc
assertItemsEqual was removed from Python's unittest.TestCase in Python 3.3 [1][2]. We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Change-Id: I1d77929d26af29e5d060eefdac85a5da2dd145d6
* Update python-tackerclient from branch 'master' - Replace assertItemsEqual with assertCountEqual assertItemsEqual was removed from Python's unittest.TestCase in Python 3.3 [1][2]. We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Change-Id: I1d77929d26af29e5d060eefdac85a5da2dd145d6
assertItemsEqual was removed from Python's unittest.TestCase in Python 3.3 [1][2]. We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Change-Id: I631fb791ece68aac607aeae56ce96a0d472964c8
* Update osc-lib from branch 'master' - Replace assertItemsEqual with assertCountEqual assertItemsEqual was removed from Python's unittest.TestCase in Python 3.3 [1][2]. We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Change-Id: I631fb791ece68aac607aeae56ce96a0d472964c8
assertItemsEqual was removed from Python's unittest.TestCase in Python 3.3 [1][2]. We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Change-Id: I53f758e7e8870b43b675f4c63d986f07d199c259
* Update heat-translator from branch 'master' - Merge "Replace assertItemsEqual with assertCountEqual" - Replace assertItemsEqual with assertCountEqual assertItemsEqual was removed from Python's unittest.TestCase in Python 3.3 [1][2]. We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Change-Id: I53f758e7e8870b43b675f4c63d986f07d199c259
I'm not sure why travis has stopped reporting here, but now that we've dropped support for python 3.4 and python 2.7 this is passing: https://travis-ci.org/github/testing-cabal/testtools/builds/671013953 |
Maybe disabling and re-enabling the Travis integration will fix it: https://twitter.com/pganssle/status/1246437422726107137 |
perhaps it would be worth switching from travis to github actions? |
Please see PR #290 to add testing on GHA. |
assertItemsEqual was removed from Python's unittest.TestCase in Python 3.3 [1][2]. We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. Credits to [5]. [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 [5] - https://review.opendev.org/#/c/732393/ Change-Id: Ib1db7694a8f0f59d8762b02acbb4ef16e5176098
* Update keystoneauth from branch 'master' - Merge "Replace assertItemsEqual with assertCountEqual" - Replace assertItemsEqual with assertCountEqual assertItemsEqual was removed from Python's unittest.TestCase in Python 3.3 [1][2]. We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. Credits to [5]. [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 [5] - https://review.opendev.org/#/c/732393/ Change-Id: Ib1db7694a8f0f59d8762b02acbb4ef16e5176098
assertItemsEqual was removed from Python's unittest.TestCase in Python 3.3 [1][2]. We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Change-Id: I1ad0da8deda3a8cbec384b5a9c88860a526eb48c
* Update python-openstackclient from branch 'master' - Merge "Replace assertItemsEqual with assertCountEqual" - Replace assertItemsEqual with assertCountEqual assertItemsEqual was removed from Python's unittest.TestCase in Python 3.3 [1][2]. We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Change-Id: I1ad0da8deda3a8cbec384b5a9c88860a526eb48c
assertItemsEqual was removed from Python's unittest.TestCase in Python 3.3 [1][2]. We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Change-Id: I0247031614ae75c1fd9f93898b7fb57838eb593f
* Update openstack-health from branch 'master' to f489fbe8ae8be4ba01a29aa2dab95a28e04f4044 - Merge "Replace assertItemsEqual with assertCountEqual" - Replace assertItemsEqual with assertCountEqual assertItemsEqual was removed from Python's unittest.TestCase in Python 3.3 [1][2]. We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Change-Id: I0247031614ae75c1fd9f93898b7fb57838eb593f
* Update neutron from branch 'master' to 309639ca892bd50cbcf27e4373aa65984c635ea5 - Merge "Replace assertItemsEqual with assertCountEqual" - Replace assertItemsEqual with assertCountEqual assertItemsEqual was removed from Python's unittest.TestCase in Python 3.3 [1][2]. We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. NOTE(dmllr): added hacking check [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Change-Id: I7c20fec08e5dc9f67b34100c925ea6724bbd25f0
assertItemsEqual was removed from Python's unittest.TestCase in Python 3.3 [1][2]. We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. NOTE(dmllr): added hacking check [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Change-Id: I7c20fec08e5dc9f67b34100c925ea6724bbd25f0
assertItemsEqual was removed from Python's unittest.TestCase in Python 3.3 [1][2]. We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Signed-off-by: Takashi Natsume <takanattie@gmail.com> Change-Id: I485b6f1c44185eaf327aadb044d6d2b1dce40eb2
* Update requirements from branch 'master' to 7a55d689e00d92de938a5c9077719faf8f980daf - Replace assertItemsEqual with assertCountEqual assertItemsEqual was removed from Python's unittest.TestCase in Python 3.3 [1][2]. We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Signed-off-by: Takashi Natsume <takanattie@gmail.com> Change-Id: I485b6f1c44185eaf327aadb044d6d2b1dce40eb2
@mtreinish could you make a release with this change? The latest release is broken on Python 3.10 because of unittest2. |
@dtantsur I would, but I unfortunately do not have access to the project on pypi otherwise I would have released already. I'll try to reach out to @rbtcollins and/or @jelmer and fix that so we can get the release out ahead of the 3.10 release |
Great to know, thanks! FYI Fedora Rawhide has already switched to 3.10, so everything using testtools may fail to build. |
@mtreinish If you do not get an answer quickly you can raise a ticket on https://github.com/pypa/pypi-support/issues/new/choose asking to be added as owner. You can also mention me in the ticket and I will back your request. Usually pypa folks are quite fast on helping with tickets like this. |
@ssbarnea they are responsive, but they also don't take over non-orphaned projects. testtools has 5 people capable of doing releases - me; @freeekanayaka @jelmer @jml @thomir :- perhaps some are no longer interested in the project in which case they can remove themselves. @mtreinish you're mtreinish on pypi ? |
@rbtcollins yeah, that's my pypi username. (https://pypi.org/user/mtreinish/ ) |
assertItemsEqual was removed from Python's unittest.TestCase in Python 3.3 [1][2]. We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Change-Id: I1a4a9c9a5497c96fe7956d045edc574f81cc5860
@mtreinish done |
assertItemsEqual was removed from Python's unittest.TestCase in Python 3.3 [1][2]. We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Change-Id: I0bbffbec8889b8b3067cfe17d258f5cb16624f38
* Update python-openstackclient from branch 'master' to 4fc1ac2ce24478f65d58face3d28d5471071cb54 - Merge "Replace assertItemsEqual with assertCountEqual" - Replace assertItemsEqual with assertCountEqual assertItemsEqual was removed from Python's unittest.TestCase in Python 3.3 [1][2]. We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Change-Id: I0bbffbec8889b8b3067cfe17d258f5cb16624f38
assertItemsEqual was removed from Python's unittest.TestCase in Python 3.3 [1][2]. We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. NOTE(dmllr): added hacking check [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Change-Id: I7c20fec08e5dc9f67b34100c925ea6724bbd25f0 (cherry picked from commit 62729d4)
assertItemsEqual was removed from Python's unittest.TestCase in Python 3.3 [1][2]. We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Conflicts: openstackclient/tests/unit/volume/v2/test_volume_snapshot.py NOTE(jpena): This file is not present in stable/wallaby Change-Id: I0bbffbec8889b8b3067cfe17d258f5cb16624f38 (cherry picked from commit e82a058)
* Update keystone from branch 'master' to a9fa5131cda296743e4916577ed968b488a414d6 - Merge "Replace assertItemsEqual with assertCountEqual" - Replace assertItemsEqual with assertCountEqual assertItemsEqual was removed from Python's unittest.TestCase in Python 3.3 [1][2]. We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Change-Id: I7725cead76c0c7349af9a8c8e8a54290caebce9c
assertItemsEqual was removed from Python's unittest.TestCase in Python 3.3 [1][2]. We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Change-Id: I7725cead76c0c7349af9a8c8e8a54290caebce9c
assertItemsEqual was removed from Python's unittest.TestCase in Python 3.3 [1][2]. We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Change-Id: I2edc09748de1739c558040a8ae6a15373ad1a93b
* Update keystone-tempest-plugin from branch 'master' to f57af91f9a97a822d2deee1fd32857ad4571d210 - Merge "Replace assertItemsEqual with assertCountEqual" - Replace assertItemsEqual with assertCountEqual assertItemsEqual was removed from Python's unittest.TestCase in Python 3.3 [1][2]. We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Change-Id: I2edc09748de1739c558040a8ae6a15373ad1a93b
* Update yaql from branch 'master' to 027f18133c3ecc03b27af13f3ef59c40b077fd37 - Merge "Replace assertItemsEqual with assertCountEqual" - Replace assertItemsEqual with assertCountEqual assertItemsEqual was removed from Python's unittest.TestCase in Python 3.3 [1][2]. We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Change-Id: I6ebd93aa093caad6f2d3db6e43beb8bc8d9cedb7
assertItemsEqual was removed from Python's unittest.TestCase in Python 3.3 [1][2]. We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Change-Id: I6ebd93aa093caad6f2d3db6e43beb8bc8d9cedb7
Patch Set 9: It looks like the grenade job is failing because of a testtools issue: testing-cabal/testtools#272 (well the reverse case of what I documented there unittest run with a testtools suite vs testtools run with a unittest suite). The stestr release added here, 2.5.0[1], switched the internal test runner it was using to be stdlib unittest based for python 3 instead of testtools based. This was done to fix a myriad of bugs for people who don't use testtools (which outside of openstack is no widely used) and was a constant source of bugs and complaints from non-openstack stestr users. I'm not sure what the best path forward to fix that bug is. I can't really revert the stestr internal test runner change because it was necessary to fix critical issues for other projects. Testtools is in a pretty stale state because of a lack of maintenance (I was recently given maintainer privileges on it, but lack the time to really do anything with it) it's unlikely to be fixed over there anytime soon. I do have an ideal long term fix proposed here: testing-cabal/testtools#277 but that can't merge until python2 and python 3.4 support is dropped from testtools. Maybe we can come up with a smaller more self contained fix to workaround unittest2 not playing well with stdlib unittest in newer python 3 and get that merged in testtools and I can push out a quick maint. release. [1] https://github.com/mtreinish/stestr/releases/tag/2.5.0 Patch-set: 9 Reviewer: Gerrit User 5196 <5196@4a232e18-c5a9-48ee-94c0-e04e7cca6543> Label: Verified=0
Currently testtools bases all of it's unittest extensions off of
unittest2 instead of the stdlib unittest. At one point this made sense
since unittest2 provided a stable base as unittest in stdlib added
features. But it's been ~5 years since there was a unittest2 release
(or a patch merged) and things have changed since then. The best
example of this is of the supported python versions listed in the
unittest2 project description/README only one is still supported by
upstream python, 2.7, which goes end of life at the end of this year.
More specific to testtools the use of unittest2 causes a whole slew of
issues because of differences in behavior with stdlib unittest. For
example here a couple issues encountered:
https://bugs.launchpad.net/testtools/+bug/1467558
https://bugs.launchpad.net/testtools/+bug/1417803
mtreinish/stestr#238
#272
which are caused, at least in part, by unittest2. There are likely other
bugs related to it that haven't been reported (or I just missed/forgot
about). At this point it's better to remove the unittest2 usage and just
rely on the upstream stdlib unittest which if nothing else is actively
maintained. It'll improve compatibility using the testtools runner with
stdlib unittest test suites and removes the class of bugs caused by
the differences in unittest2.
Fixes #263