Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test_venv: test_zippath_from_non_installed_posix() failed on aarch64 RHEL8 Refleaks 3.x #109748

Closed
vstinner opened this issue Sep 22, 2023 · 1 comment
Closed

test_venv: test_zippath_from_non_installed_posix() failed on aarch64 RHEL8 Refleaks 3.x #109748

vstinner opened this issue Sep 22, 2023 · 1 comment
Labels
tests Tests in the Lib/test dir

Comments

@vstinner
Copy link
Member

vstinner commented Sep 22, 2023 •
edited by bedevere-app bot

cp874.cpython-313.pyc.281473310212544 looks like a temporary PYC filename used by importlib to create PYC files in atomic way.

The problem seems to be that tests are run in parallel and that files can appear or disappear while test_venv is running (since other tests are running at the same time).

I'm not sure why test_zippath_from_non_installed_posix() wants to eagerly copies __pycache__/ directories. Is it important to copy PYC files for a "zipapp"?

aarch64 RHEL8 Refleaks 3.x:

0:00:14 load avg: 8.78 [ 68/463/1] test_venv failed (1 error)
beginning 6 repetitions
123456
Could not find platform dependent libraries <exec_prefix>
.Could not find platform dependent libraries <exec_prefix>
.test test_venv failed -- Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.refleak/build/Lib/test/test_venv.py", line 578, in test_zippath_from_non_installed_posix
    shutil.copytree(fn, os.path.join(libdir, name))
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.refleak/build/Lib/shutil.py", line 588, in copytree
    return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.refleak/build/Lib/shutil.py", line 542, in _copytree
    raise Error(errors)
shutil.Error: [('/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.refleak/build/Lib/encodings/__pycache__/cp874.cpython-313.pyc.281473310212544', '/tmp/test_python_fw_naiq7/tmpfycz07rf/lib/python3.13/encodings/__pycache__/cp874.cpython-313.pyc.281473310212544', "[Errno 2] No such file or directory: '/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.refleak/build/Lib/encodings/__pycache__/cp874.cpython-313.pyc.281473310212544'")]

build: https://buildbot.python.org/all/#/builders/551/builds/865

By the way, Could not find platform dependent libraries <exec_prefix> message is surprising.

Linked PRs
@vstinner vstinner added the tests Tests in the Lib/test dir label Sep 22, 2023
@vstinner
Copy link
Member Author

aarch64 Fedora Stable Refleaks 3.x: https://buildbot.python.org/all/#/builders/123/builds/857

vstinner added a commit to vstinner/cpython that referenced this issue Sep 25, 2023
@vstinner
pythongh-109748: Fix venv test_zippath_from_non_installed_posix()
8120c89 
Fix test_zippath_from_non_installed_posix() of test_venv: don't copy
__pycache__/ directories because they can be modified by other Python
tests running in parallel.
vstinner added a commit to vstinner/cpython that referenced this issue Sep 25, 2023
@vstinner
pythongh-109748: Fix venv test_zippath_from_non_installed_posix()
26c30da 
Fix test_zippath_from_non_installed_posix() of test_venv: don't copy
__pycache__/ directories because they can be modified by other Python
tests running in parallel.
vstinner added a commit to vstinner/cpython that referenced this issue Sep 25, 2023
@vstinner
pythongh-109748: Fix venv test_zippath_from_non_installed_posix()
338ea4e 
Fix test_zippath_from_non_installed_posix() of test_venv: don't copy
__pycache__/ directories because they can be modified by other Python
tests running in parallel.
vstinner added a commit to vstinner/cpython that referenced this issue Sep 25, 2023
@vstinner
pythongh-109748: Fix venv test_zippath_from_non_installed_posix()
374e0fe 
Fix test_zippath_from_non_installed_posix() of test_venv: don't copy
__pycache__/ sub-directories, because they can be modified by other
Python tests running in parallel.
@bedevere-app bedevere-app bot mentioned this issue Sep 25, 2023
Merged
vstinner added a commit that referenced this issue Sep 25, 2023
@vstinner
gh-109748: Fix venv test_zippath_from_non_installed_posix() (#109872)
25bb266 
Fix test_zippath_from_non_installed_posix() of test_venv: don't copy
__pycache__/ sub-directories, because they can be modified by other
Python tests running in parallel.
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Sep 25, 2023
@vstinner @miss-islington
pythongh-109748: Fix venv test_zippath_from_non_installed_posix() (py…
fab06b4 
…thonGH-109872)

Fix test_zippath_from_non_installed_posix() of test_venv: don't copy
__pycache__/ sub-directories, because they can be modified by other
Python tests running in parallel.
(cherry picked from commit 25bb266)

Co-authored-by: Victor Stinner <vstinner@python.org>
@bedevere-app bedevere-app bot mentioned this issue Sep 25, 2023
Closed
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Sep 25, 2023
@vstinner @miss-islington
pythongh-109748: Fix venv test_zippath_from_non_installed_posix() (py…
80f594c 
…thonGH-109872)

Fix test_zippath_from_non_installed_posix() of test_venv: don't copy
__pycache__/ sub-directories, because they can be modified by other
Python tests running in parallel.
(cherry picked from commit 25bb266)

Co-authored-by: Victor Stinner <vstinner@python.org>
@bedevere-app bedevere-app bot mentioned this issue Sep 25, 2023
Merged
vstinner added a commit that referenced this issue Sep 25, 2023
@miss-islington @vstinner
[3.11] gh-109748: Fix venv test_zippath_from_non_installed_posix() (G…
358282e 
…H-109872) (#109874)

gh-109748: Fix venv test_zippath_from_non_installed_posix() (GH-109872)

Fix test_zippath_from_non_installed_posix() of test_venv: don't copy
__pycache__/ sub-directories, because they can be modified by other
Python tests running in parallel.
(cherry picked from commit 25bb266)

Co-authored-by: Victor Stinner <vstinner@python.org>
@vstinner vstinner mentioned this issue Sep 27, 2023
Closed
csm10495 pushed a commit to csm10495/cpython that referenced this issue Sep 28, 2023
@vstinner @csm10495
pythongh-109748: Fix venv test_zippath_from_non_installed_posix() (py…
fe9f2a8 
…thon#109872)

Fix test_zippath_from_non_installed_posix() of test_venv: don't copy
__pycache__/ sub-directories, because they can be modified by other
Python tests running in parallel.
vstinner added a commit to vstinner/cpython that referenced this issue Sep 30, 2023
@vstinner
pythongh-109748: Fix again venv test_zippath_from_non_installed_posix()
52fc8c4 
Call also copy_python_src_ignore() on listdir() names.

shutil.copytree(): replace set() with an empty tuple. An empty tuple
becomes a constant in the compiler and checking if an item is in an
empty tuple is cheap.
@bedevere-app bedevere-app bot mentioned this issue Sep 30, 2023
Merged
vstinner added a commit that referenced this issue Sep 30, 2023
@vstinner
gh-109748: Fix again venv test_zippath_from_non_installed_posix() (#1…
0def8c7 
…10149)

Call also copy_python_src_ignore() on listdir() names.

shutil.copytree(): replace set() with an empty tuple. An empty tuple
becomes a constant in the compiler and checking if an item is in an
empty tuple is cheap.
@bedevere-app bedevere-app bot mentioned this issue Sep 30, 2023
Merged
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Sep 30, 2023
@vstinner @miss-islington
pythongh-109748: Fix again venv test_zippath_from_non_installed_posix…
c9c567f 
…() (pythonGH-110149)

Call also copy_python_src_ignore() on listdir() names.

shutil.copytree(): replace set() with an empty tuple. An empty tuple
becomes a constant in the compiler and checking if an item is in an
empty tuple is cheap.
(cherry picked from commit 0def8c7)

Co-authored-by: Victor Stinner <vstinner@python.org>
vstinner added a commit that referenced this issue Sep 30, 2023
@miss-islington @vstinner
[3.11] gh-109748: Fix again venv test_zippath_from_non_installed_posi…
080f5b9 
…x() (GH-110149) (#110153)

gh-109748: Fix again venv test_zippath_from_non_installed_posix() (GH-110149)

Call also copy_python_src_ignore() on listdir() names.

shutil.copytree(): replace set() with an empty tuple. An empty tuple
becomes a constant in the compiler and checking if an item is in an
empty tuple is cheap.
(cherry picked from commit 0def8c7)

Co-authored-by: Victor Stinner <vstinner@python.org>
vstinner added a commit to vstinner/cpython that referenced this issue Oct 4, 2023
@vstinner
pythongh-109748: Fix again venv test_zippath_from_non_installed_posix…
a334057 
…() (python#110149)

Call also copy_python_src_ignore() on listdir() names.

shutil.copytree(): replace set() with an empty tuple. An empty tuple
becomes a constant in the compiler and checking if an item is in an
empty tuple is cheap.

(cherry picked from commit 0def8c7)
vstinner added a commit that referenced this issue Oct 4, 2023
@vstinner @zooba
[3.12] gh-109615: Fix support test_copy_python_src_ignore() (#109958) (…
8939221 
…#110340)

* gh-109615: Fix support test_copy_python_src_ignore() (#109958)

Fix the test when run on an installed Python: use "abs_srcdir" of
sysconfig, and skip the test if the Python source code cannot be
found.

* Tools/patchcheck/patchcheck.py, Tools/freeze/test/freeze.py and
  Lib/test/libregrtest/utils.py now first try to get "abs_srcdir"
  from sysconfig, before getting "srcdir" from sysconfig.
* test.pythoninfo logs sysconfig "abs_srcdir".

(cherry picked from commit b89ed9d)

* gh-109615: Fix support test_copy_python_src_ignore() on WASM (#109970)

Not only check if src_dir exists, but look also for Lib/os.py
landmark.

(cherry picked from commit cc54bcf)

* gh-109615: Look for 'Modules' as landmark for test_copy_python_src_ignore (GH-110108)

(cherry picked from commit 20bc5f7)

* gh-109748: Fix again venv test_zippath_from_non_installed_posix() (#110149)

Call also copy_python_src_ignore() on listdir() names.

shutil.copytree(): replace set() with an empty tuple. An empty tuple
becomes a constant in the compiler and checking if an item is in an
empty tuple is cheap.

(cherry picked from commit 0def8c7)

---------

Co-authored-by: Steve Dower <steve.dower@python.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
tests Tests in the Lib/test dir
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@vstinner