Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ERROR: test_sundry (test.test_tools.test_sundry.TestSundryScripts): SystemExit: ERROR: missing _freeze_module #89565

Closed
hroncok mannequin opened this issue Oct 7, 2021 · 7 comments
Labels
3.11 only security fixes tests Tests in the Lib/test dir

Comments

@hroncok
Copy link
Mannequin

hroncok mannequin commented Oct 7, 2021

BPO 45402
Nosy @vstinner, @hroncok, @pablogsal
PRs
  • bpo-45402: Fix test_tools.test_sundry() #28786
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2021-10-07.19:19:53.416>
    created_at = <Date 2021-10-07.10:05:28.827>
    labels = ['tests', '3.11']
    title = 'ERROR: test_sundry (test.test_tools.test_sundry.TestSundryScripts): SystemExit: ERROR: missing _freeze_module'
    updated_at = <Date 2021-11-22.12:36:40.928>
    user = 'https://github.com/hroncok'

    bugs.python.org fields:

    activity = <Date 2021-11-22.12:36:40.928>
    actor = 'hroncok'
    assignee = 'none'
    closed = True
    closed_date = <Date 2021-10-07.19:19:53.416>
    closer = 'vstinner'
    components = ['Tests']
    creation = <Date 2021-10-07.10:05:28.827>
    creator = 'hroncok'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 45402
    keywords = ['patch']
    message_count = 7.0
    messages = ['403377', '403382', '403384', '403392', '403435', '403436', '406767']
    nosy_count = 3.0
    nosy_names = ['vstinner', 'hroncok', 'pablogsal']
    pr_nums = ['28786']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue45402'
    versions = ['Python 3.11']

    @hroncok
    Copy link
    Mannequin Author

    hroncok mannequin commented Oct 7, 2021

    When we build Python 3.10.0a1 (from the git tag) in Fedora, we see:

    ======================================================================
    ERROR: test_sundry (test.test_tools.test_sundry.TestSundryScripts)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "/builddir/build/BUILD/Python-3.11.0a1/Lib/test/test_tools/test_sundry.py", line 43, in test_sundry
        import_tool(name)
        ^^^^^^^^^^^^^^^^^
      File "/builddir/build/BUILD/Python-3.11.0a1/Lib/test/test_tools/__init__.py", line 35, in import_tool
        return importlib.import_module(toolname)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/builddir/build/BUILD/Python-3.11.0a1/Lib/importlib/__init__.py", line 126, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "<frozen importlib._bootstrap>", line 1072, in _gcd_import
      File "<frozen importlib._bootstrap>", line 1044, in _find_and_load
      File "<frozen importlib._bootstrap>", line 1015, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 689, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 894, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "/builddir/build/BUILD/Python-3.11.0a1/Tools/scripts/freeze_modules.py", line 37, in <module>
        sys.exit("ERROR: missing _freeze_module")
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    SystemExit: ERROR: missing _freeze_module

    Ran 3 tests in 0.209s

    I see in the build log we use Programs/_freeze_module regularly in the build process, so no idea why this fails. I'll try to come up with a reporducer.

    @hroncok hroncok mannequin added 3.11 only security fixes tests Tests in the Lib/test dir labels Oct 7, 2021
    @hroncok
    Copy link
    Mannequin Author

    hroncok mannequin commented Oct 7, 2021

    I can reproduce this from git main branch:

    $ git clean -fdx
    $ mkdir -p build/debug
    $ cd build/debug
    $ ../../configure --with-platlibdir=lib64 --enable-shared --without-ensurepip --with-pydebug
    $ make
    $ cd ../..
    $ LD_LIBRARY_PATH=$PWD/build/debug build/debug/python -m test -v test_tools

    @hroncok
    Copy link
    Mannequin Author

    hroncok mannequin commented Oct 7, 2021

    I can also reproduce this without the additional configure flags:

    [cpython]$ git clean -fdx
    [cpython]$ mkdir -p build/debug
    [cpython]$ cd build/debug
    [debug]$ ../../configure --with-pydebug
    [debug]$ make
    [debug]$ cd ../..
    [cpython]$ build/debug/python -m test -v test_tools
    ...
    SystemExit: ERROR: missing _freeze_module

    As well as from the build dir:

    [cpython]$ cd build/debug
    [debug]$ ./python -m test -v test_tools
    ...
    SystemExit: ERROR: missing _freeze_module

    But when I build from the source directory directly:

    [cpython]$ git clean -fdx
    [cpython]$ ./configure --with-pydebug
    [cpython]$ make
    [cpython]$ ./python -m test -v test_tools
    ...
    Tests result: SUCCESS

    Hence, I assume the culprit is in building from a subdirectory.

    @vstinner
    Copy link
    Member

    vstinner commented Oct 7, 2021

    My fix for bpo-45020 no longer works, I'm not sure why:

    commit 41551ee
    Author: Victor Stinner <vstinner@python.org>
    Date: Fri Sep 17 20:20:43 2021 +0200

    bpo-45020: Fix build out of source tree (GH-28410)
    
    * Makefile.pre.in: Add $(srcdir) when needed, remove it when it was
      used by mistake.
    * freeze_modules.py tool uses ./Programs/_freeze_module if the
      executable doesn't exist in the source tree.
    

    I'm working on a new fix.

    @vstinner
    Copy link
    Member

    vstinner commented Oct 7, 2021

    New changeset ff8859d by Victor Stinner in branch 'main':
    bpo-45402: Fix test_tools.test_sundry() (GH-28786)
    ff8859d

    @vstinner
    Copy link
    Member

    vstinner commented Oct 7, 2021

    Thanks Miro for the bug report, it's now fixed.

    @hroncok
    Copy link
    Mannequin Author

    hroncok mannequin commented Nov 22, 2021

    There seem to be a regression in this fix, see https://bugs.python.org/issue45866

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.11 only security fixes tests Tests in the Lib/test dir
    Projects
    None yet
    Development

    No branches or pull requests

    1 participant