test_embed failure with certain installed site-packages

[kulikjak]

When running the test suite for Python 3.11, one test_embed test is failing:

======================================================================
FAIL: test_quickened_static_code_gets_unquickened_at_Py_FINALIZE (test.test_embed.EmbeddingTests.test_quickened_static_code_gets_unquickened_at_Py_FINALIZE)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/scratch/userland-gate/components/python/python311/Python-3.11.0rc1/Lib/test/test_embed.py", line 375, in test_quickened_static_code_gets_unquickened_at_Py_FINALIZE
    out, err = run("test_repeated_init_exec", code)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/scratch/userland-gate/components/python/python311/Python-3.11.0rc1/Lib/test/test_embed.py", line 115, in run_embedded_interpreter
    self.assertEqual(p.returncode, returncode,
AssertionError: 1 != 0 : bad returncode 1, stderr is '--- Loop #1 ---\nTraceback (most recent call last):\n  File "<string>", line 8, in <module>\nAssertionError\n'

I found out that it only fails when zope-interface is also installed - more specifically, it's the zope.interface-4.6.0-py3.11-nspkg.pth file generated by setuptools and installed into site-packages. When I remove it, the test passes again.

Environment:
Python 3.11.0rc1 (main, Aug 19 2022, 10:02:19) [GCC 11.3.0] on sunos5
Setuptools 62.1.0 or 65.3.0
Zope interface 4.6.0
(and many other packages)

This is on Oracle Solaris, and I am not sure whether it's happening on other platforms as well. I am also not sure that this is necessarily CPython issue - I guess it might be setuptools as well?

[sweeneyde]

Looks like this is the failing program:

import importlib._bootstrap
func = importlib._bootstrap._handle_fromlist
code = func.__code__

# Assert initially unquickened.
# Use sets to account for byte order.
if set(code._co_code_adaptive[:2]) != set([{resume}, 0]):
    raise AssertionError()  # <=====================================

Presumably, the zope-interface package is doing some imports and warming up the importlib._bootstrap._handle_fromlist function, so the assertion fails.

Perhaps there is some parameter that can be passed to self.run_embedded_interpreter to not look at site-packages, like -I.

[sweeneyde]

@vstinner wrote much of the _testembed.c code. Any ideas to make sure the embedded interpreter runs without site-packages?

[sweeneyde]

Another option would be to use a different, less-likely-to-be-used function from some other deepfrozen module, or maybe even introduce another private, otherwise unused function to some deepfrozen module. test.__phello__.spam may be a good candidate.

[vstinner]

I fixed an issue about pth files in test_site: commit d18de46. The fix skips the test if a pth is found. Maybe test_embed should do the same and the code should be moved somewhere in test.support.

[vstinner]

Is there a way to opt out from pth files in Python? Are pth files ignored when using the -I (isolated) option?

[ericsnowcurrently]

Is there a way to opt out from pth files in Python? Are pth files ignored when using the -I (isolated) option?

The only option that prevents loading pth files is '-S' (and -s).

@warsaw

[krunch3r76]

this appears to be fixed upstream on the main branch at 65c2850 but i want to add that it is definitely an issue with the user site-packages directory at the current release 3.11.5 as i get the same failure on test_embed (lines 115, 375) but if i remove ~/.local/lib/python3.11/site-packages/backports.csv-1.0.7-py3.7-nspkg.pth the test passes.

while zope.interface-4.6.0-py3.11-nspkg.pth is not in there, other .pth files under site-packages/ exist but do not cause the test to fail. it seems the test is not fully isolated from the user and is picking up .pth files inadvertently. note, i do not specify ~/.local/lib in any of my environment variables.

Environment:

• linux (from scratch 11.3) as non-admin user with locally installed packages under ~/.local/lib/python3.11/site-packages
• python 3.11.5 release source tree (also 3.11.2)

Reproduce with:

(./Python-3.11.5) $ pip3 install backports.csv==1.0.7
(./Python-3.11.5) $ ./configure && make && make test

Relevant output:

:00:03 load avg: 3.69 [146/434/2] test_embed failed (1 failure)
test test_embed failed -- Traceback (most recent call last):
  File "/home/krunch3r/tmp/Python-3.11.5/Lib/test/test_embed.py", line 381, in test_quickened_static_code_gets_unquickened_at_Py_FINALIZE
    out, err = run("test_repeated_init_exec", code)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/krunch3r/tmp/Python-3.11.5/Lib/test/test_embed.py", line 115, in run_embedded_interpreter
    self.assertEqual(p.returncode, returncode,
AssertionError: 1 != 0 : bad returncode 1, stderr is '--- Loop #108227 
Traceback (most recent call last):\n  File "<string>", line 8, in <module>
AssertionError`

[vstinner]

@ericsnowcurrently:

The only option that prevents loading pth files is '-S' (and -s).

Does someone want to propose a change to pass this flag / enable the option in affected tests?