test_lib2to3 fails since Python 3.9 #86136

felixonmars · 2020-10-07T23:05:58Z

BPO	41970
Nosy	@felixonmars, @pablogsal, @miss-islington, @tirkarthi
PRs	bpo-41970: Avoid test failure in test_lib2to3 if the module is already imported #22595 [3.9] bpo-41970: Avoid test failure in test_lib2to3 if the module is already imported (GH-22595) #22609

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-10-09.10:09:01.376>
created_at = <Date 2020-10-07.23:05:57.966>
labels = ['tests', '3.9', '3.10']
title = 'test_lib2to3 fails since Python 3.9'
updated_at = <Date 2020-10-09.10:09:01.375>
user = 'https://github.com/felixonmars'

bugs.python.org fields:

activity = <Date 2020-10-09.10:09:01.375>
actor = 'pablogsal'
assignee = 'none'
closed = True
closed_date = <Date 2020-10-09.10:09:01.376>
closer = 'pablogsal'
components = ['Tests']
creation = <Date 2020-10-07.23:05:57.966>
creator = 'felixonmars'
dependencies = []
files = []
hgrepos = []
issue_num = 41970
keywords = ['patch']
message_count = 14.0
messages = ['378190', '378196', '378205', '378208', '378232', '378234', '378238', '378240', '378241', '378242', '378245', '378246', '378249', '378310']
nosy_count = 5.0
nosy_names = ['felixonmars', 'pablogsal', 'miss-islington', 'xtreak', 'federico.granata']
pr_nums = ['22595', '22609']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue41970'
versions = ['Python 3.9', 'Python 3.10']

felixonmars · 2020-10-07T23:05:58Z

I am packaging Python for Arch and the tests suite of Python 3.8.6 pass here without this:

0:09:06 load avg: 0.87 [205/424] test_lib2to3                                                                                                                                                                                                                                                                         
test test_lib2to3 crashed -- Traceback (most recent call last):                                                                                                                                                                                                                                                       
  File "/build/python/src/Python-3.9.0/Lib/test/libregrtest/runtest.py", line 270, in _runtest_inner                                                                                                                                                                                                                  
    refleak = _runtest_inner2(ns, test_name)                                                                                                                                                                                                                                                                          
  File "/build/python/src/Python-3.9.0/Lib/test/libregrtest/runtest.py", line 221, in _runtest_inner2                                                                                                                                                                                                                 
    the_module = importlib.import_module(abstest)                                                                                                                                                                                                                                                                     
  File "/build/python/src/Python-3.9.0/Lib/importlib/__init__.py", line 127, in import_module                                                                                                                                                                                                                         
    return _bootstrap._gcd_import(name[level:], package, level)                                                                                                                                                                                                                                                       
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import                                                                                                                                                                                                                                                     
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load                                                                                                                                                                                                                                                  
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked                                                                                                                                                                                                                                          
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked                                                                                                                                                                                                                                                   
  File "<frozen importlib._bootstrap_external>", line 790, in exec_module                                                                                                                                                                                                                                             
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed                                                                                                                                                                                                                                        
  File "/build/python/src/Python-3.9.0/Lib/test/test_lib2to3.py", line 5, in <module>                                                                                                                                                                                                                                 
    from lib2to3.tests import load_tests                                                                                                                                                                                                                                                                              
  File "/build/python/src/Python-3.9.0/Lib/contextlib.py", line 124, in __exit__                                                                                                                                                                                                                                      
    next(self.gen)                                                                                                                                                                                                                                                                                                    
  File "/build/python/src/Python-3.9.0/Lib/test/support/__init__.py", line 1171, in _filterwarnings                                                                                                                                                                                                                   
    raise AssertionError("filter (%r, %s) did not catch any warning" %                                                                                                                                                                                                                                                
AssertionError: filter ('', PendingDeprecationWarning) did not catch any warning                                                                                                                                                                                                                                      
                                                                                                                                                                                                                                                                                                                      
0:09:06 load avg: 0.87 [206/424/1] test_linecache -- test_lib2to3 failed

pablogsal · 2020-10-07T23:31:12Z

Same thing as in https://bugs.python.org/issue41971. I cannot reproduce this with a fresh install of 3.9 and our buildbots are not complaining about this.

ownloads/Python-3.9.0
❯ ./python -m test test_lib2to3
0:00:00 load avg: 3.64 Run tests sequentially
0:00:00 load avg: 3.64 [1/1] test_lib2to3

== Tests result: SUCCESS ==

1 test OK.

Total duration: 7.5 sec
Tests result: SUCCESS

-------

❯ ./lel/bin/python3 -m test test_lib2to3
0:00:00 load avg: 3.87 Run tests sequentially
0:00:00 load avg: 3.87 [1/1] test_lib2to3

== Tests result: SUCCESS ==

1 test OK.

Total duration: 7.4 sec
Tests result: SUCCESS

tirkarthi · 2020-10-08T00:15:16Z

I have also seen this and linked issue on running whole test suite while building python myself but running individual tests show no error and buildbots also never complained so I thought it's something wrong with my setup. I introduced the commit to silence deprecation warning due to lib2to3 and cannot reproduce the same running it individually.

pablogsal · 2020-10-08T00:24:42Z

I have also seen this and linked issue on running whole test suite while building python myself but running individual tests show no error and buildbots also never complained so I thought it's something wrong with my setup. I introduced the commit to silence deprecation warning due to lib2to3 and cannot reproduce the same running it individually.

What I don't understand then is how the buildbots or the CI in Github that run all tests sequentially with the extra flags (-uall and friends) have not complained yet about this.

@felix, how consistent is this failure you are experiencing?

federicogranata · 2020-10-08T10:57:30Z

I just experienced the very same issue.
I use pyenv and doing all tests gives me a failure on test_lib2to3 while running it individually end succesfully.

Tell me if I can provide any useful information

tirkarthi · 2020-10-08T11:17:39Z

Pablo, I can see the error in Travis. The difference I can see is that the tests are executed sequentially in coverage run but normal run has tests executing in parallel. Felix and Federico, can you please add the command you use to run the tests?

Sample error build : https://travis-ci.com/github/python/cpython/jobs/396688564#L2022

coverage run but has true in the end thus being always successful

xvfb-run ./venv/bin/python -m coverage run --branch --pylib -m test --fail-env-changed -uall,-cpu -x test_multiprocessing_fork -x test_multiprocessing_forkserver -x test_multiprocessing_spawn -x test_concurrent_futures || true

Normal run

/home/travis/build/python/cpython/python -u -W default -bb -E -m test -r -w -j 1 -u all -W --slowest --fail-env-changed --timeout=1200 -j4 -uall,-cpu

felixonmars · 2020-10-08T11:36:07Z

I use this in Arch's packaging:

LC_CTYPE=en_US.UTF-8 xvfb-run -s "-screen 0 1920x1080x16 -ac +extension GLX" -a -n "$servernum" \
    "${srcdir}/Python-${pkgver}/python" -m test.regrtest -v -uall -x test_tk

(test_tk is currently skipped due to https://bugs.python.org/issue41306)

federicogranata · 2020-10-08T12:02:14Z

I just run "python -m test" for it to fail and "python -m test test_lib2to3" for it to succeded.

tirkarthi · 2020-10-08T12:13:56Z

I can reproduce this by running test_lib2to3 twice or running test___all__ with test_lib2to3

./python -m test test_lib2to3 test_lib2to3
0:00:00 load avg: 0.03 Run tests sequentially
0:00:00 load avg: 0.03 [1/2] test_lib2to3
0:00:13 load avg: 0.25 [2/2] test_lib2to3
test test_lib2to3 crashed -- Traceback (most recent call last):
  File "/root/cpython/Lib/test/libregrtest/runtest.py", line 272, in _runtest_inner
    refleak = _runtest_inner2(ns, test_name)
  File "/root/cpython/Lib/test/libregrtest/runtest.py", line 223, in _runtest_inner2
    the_module = importlib.import_module(abstest)
  File "/root/cpython/Lib/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 790, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/root/cpython/Lib/test/test_lib2to3.py", line 5, in <module>
    from lib2to3.tests import load_tests
  File "/root/cpython/Lib/contextlib.py", line 124, in __exit__
    next(self.gen)
  File "/root/cpython/Lib/test/support/warnings_helper.py", line 179, in _filterwarnings
    raise AssertionError("filter (%r, %s) did not catch any warning" %
AssertionError: filter ('', PendingDeprecationWarning) did not catch any warning

test_lib2to3 failed

== Tests result: FAILURE ==

1 test OK.

1 test failed:
test_lib2to3

Total duration: 13.5 sec
Tests result: FAILURE

./python -m test test___all__ test_lib2to3     
0:00:00 load avg: 0.05 Run tests sequentially
0:00:00 load avg: 0.05 [1/2] test___all__
0:00:00 load avg: 0.05 [2/2] test_lib2to3
test test_lib2to3 crashed -- Traceback (most recent call last):
  File "/root/cpython/Lib/test/libregrtest/runtest.py", line 272, in _runtest_inner
    refleak = _runtest_inner2(ns, test_name)
  File "/root/cpython/Lib/test/libregrtest/runtest.py", line 223, in _runtest_inner2
    the_module = importlib.import_module(abstest)
  File "/root/cpython/Lib/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 790, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/root/cpython/Lib/test/test_lib2to3.py", line 5, in <module>
    from lib2to3.tests import load_tests
  File "/root/cpython/Lib/contextlib.py", line 124, in __exit__
    next(self.gen)
  File "/root/cpython/Lib/test/support/warnings_helper.py", line 179, in _filterwarnings
    raise AssertionError("filter (%r, %s) did not catch any warning" %
AssertionError: filter ('', PendingDeprecationWarning) did not catch any warning

test_lib2to3 failed

== Tests result: FAILURE ==

1 test OK.

1 test failed:
test_lib2to3

Total duration: 1.1 sec
Tests result: FAILURE

federicogranata · 2020-10-08T12:17:11Z

I can have the test to fails if I run "python -m test test_lib2to3 test_lib2to3" as well

pablogsal · 2020-10-08T12:41:06Z

Ok, thanks everyone for the help reproducing this. I think I can work on a fix today

tirkarthi · 2020-10-08T12:55:15Z

I guess this could be due to loading the module twice where the warning is not emitted again on reimport which test__all__ seems to do so by importing lib2to3. One obvious fix would be to pass quiet=True. Else we need to find a way to import the module fresh like using import_fresh_module. Two possible patches. I assumed running tests sequentially will provide support for isolation too.

Optional argument:
 - if 'quiet' is True, it does not fail if a filter catches nothing
    (default True without argument,
     default False if some filters are defined)

Without argument, it defaults to:
    check_warnings(("", Warning), quiet=True)

# Using import fresh module

diff --git a/Lib/test/test_lib2to3.py b/Lib/test/test_lib2to3.py
index 159a8387e4..861ae5ad53 100644
--- a/Lib/test/test_lib2to3.py
+++ b/Lib/test/test_lib2to3.py
@@ -1,7 +1,9 @@
 import unittest
+from test.support.import_helper import import_fresh_module
 from test.support.warnings_helper import check_warnings
 
 with check_warnings(("", PendingDeprecationWarning)):
+    lib2to3 = import_fresh_module("lib2to3")
     from lib2to3.tests import load_tests
 
 if __name__ == '__main__':

# Passing quiet=True

diff --git a/Lib/test/test_lib2to3.py b/Lib/test/test_lib2to3.py
index 159a8387e4..e4c5cade8b 100644
--- a/Lib/test/test_lib2to3.py
+++ b/Lib/test/test_lib2to3.py
@@ -1,7 +1,7 @@
 import unittest
 from test.support.warnings_helper import check_warnings
 
-with check_warnings(("", PendingDeprecationWarning)):
+with check_warnings(("", PendingDeprecationWarning), quiet=True):
     from lib2to3.tests import load_tests
 
 if __name__ == '__main__':

➜ cpython git:(master) ✗ ./python -m test test_lib2to3 test_lib2to3
0:00:00 load avg: 0.01 Run tests sequentially
0:00:00 load avg: 0.01 [1/2] test_lib2to3
0:00:13 load avg: 0.23 [2/2] test_lib2to3

== Tests result: SUCCESS ==

All 2 tests OK.

Total duration: 27.3 sec
Tests result: SUCCESS

miss-islington · 2020-10-08T13:24:37Z

New changeset 4a9f82f by Pablo Galindo in branch 'master':
bpo-41970: Avoid test failure in test_lib2to3 if the module is already imported (GH-22595)
4a9f82f

pablogsal · 2020-10-09T10:08:50Z

New changeset c6f41e6 by Karthikeyan Singaravelan in branch '3.9':
[3.9] bpo-41970: Avoid test failure in test_lib2to3 if the module is already imported (GH-22595) (GH-22609)
c6f41e6

felixonmars mannequin added 3.9 only security fixes tests Tests in the Lib/test dir labels Oct 7, 2020

tirkarthi added 3.10 only security fixes labels Oct 8, 2020

pablogsal closed this as completed Oct 9, 2020

ezio-melotti transferred this issue from another repository Apr 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test_lib2to3 fails since Python 3.9 #86136

test_lib2to3 fails since Python 3.9 #86136

felixonmars mannequin commented Oct 7, 2020

felixonmars mannequin commented Oct 7, 2020

pablogsal commented Oct 7, 2020

tirkarthi commented Oct 8, 2020

pablogsal commented Oct 8, 2020

federicogranata mannequin commented Oct 8, 2020

tirkarthi commented Oct 8, 2020

felixonmars mannequin commented Oct 8, 2020

federicogranata mannequin commented Oct 8, 2020

tirkarthi commented Oct 8, 2020

federicogranata mannequin commented Oct 8, 2020

pablogsal commented Oct 8, 2020

tirkarthi commented Oct 8, 2020

miss-islington commented Oct 8, 2020

pablogsal commented Oct 9, 2020

test_lib2to3 fails since Python 3.9 #86136

test_lib2to3 fails since Python 3.9 #86136

Comments

felixonmars mannequin commented Oct 7, 2020

felixonmars mannequin commented Oct 7, 2020

pablogsal commented Oct 7, 2020

tirkarthi commented Oct 8, 2020

pablogsal commented Oct 8, 2020

federicogranata mannequin commented Oct 8, 2020

tirkarthi commented Oct 8, 2020

felixonmars mannequin commented Oct 8, 2020

federicogranata mannequin commented Oct 8, 2020

tirkarthi commented Oct 8, 2020

federicogranata mannequin commented Oct 8, 2020

pablogsal commented Oct 8, 2020

tirkarthi commented Oct 8, 2020

miss-islington commented Oct 8, 2020

pablogsal commented Oct 9, 2020