Tests failing on Python 3.12

[jaraco]

No description provided.

[jaraco]

One of the failing tests:

________________________________________________________________ TestBuildCLib.test_run ________________________________________________________________

self = <distutils.tests.test_build_clib.TestBuildCLib object at 0x106a09d60>

    @pytest.mark.skipif('platform.system() == "Windows"')
    def test_run(self):
        pkg_dir, dist = self.create_dist()
        cmd = build_clib(dist)
    
        foo_c = os.path.join(pkg_dir, 'foo.c')
        self.write_file(foo_c, 'int main(void) { return 1;}\n')
        cmd.libraries = [('foo', {'sources': [foo_c]})]
    
        build_temp = os.path.join(pkg_dir, 'build')
        os.mkdir(build_temp)
        cmd.build_temp = build_temp
        cmd.build_clib = build_temp
    
        # Before we run the command, we want to make sure
        # all commands are present on the system.
>       ccmd = missing_compiler_executable()

distutils/tests/test_build_clib.py:128: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

cmd_names = []

    def missing_compiler_executable(cmd_names=[]):
        """Check if the compiler components used to build the interpreter exist.
    
        Check for the existence of the compiler executables whose names are listed
        in 'cmd_names' or all the compiler executables when 'cmd_names' is empty
        and return the first missing executable or None when none is found
        missing.
    
        """
>       from setuptools._distutils import ccompiler, sysconfig, spawn
E       ModuleNotFoundError: No module named 'setuptools'

/opt/homebrew/Cellar/python@3.12/3.12.1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/test/support/__init__.py:1732: ModuleNotFoundError

It looks like the reliance on test.support has an implicit dependency on Setuptools.

[jaraco]

I tried adding setuptools; python_version >= "3.12" to the dependencies, but that doesn't work because Setuptools provides its own top-level distutils module, conflicting with this one.

[jaraco]

ff32ae0 addresses that and related failures by restoring a "missing_compiler_executable" that doesn't rely on setuptools.

[jaraco]

The other class of failure looks like:

_______________________________________________________ TestBuildPy.test_byte_compile_optimized ________________________________________________________

self = <distutils.tests.test_build_py.TestBuildPy object at 0x105247620>

    @pytest.mark.skipif('sys.dont_write_bytecode')
    def test_byte_compile_optimized(self):
        project_dir, dist = self.create_dist(py_modules=['boiledeggs'])
        os.chdir(project_dir)
        self.write_file('boiledeggs.py', 'import antigravity')
        cmd = build_py(dist)
        cmd.compile = 0
        cmd.optimize = 1
        cmd.build_lib = 'here'
        cmd.finalize_options()
>       cmd.run()

/Users/jaraco/code/pypa/distutils/distutils/tests/test_build_py.py:115: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/Users/jaraco/code/pypa/distutils/distutils/command/build_py.py:98: in run
    self.byte_compile(self.get_outputs(include_bytecode=0))
/Users/jaraco/code/pypa/distutils/distutils/command/build_py.py:400: in byte_compile
    byte_compile(
/Users/jaraco/code/pypa/distutils/distutils/util.py:462: in byte_compile
    spawn(cmd, dry_run=dry_run)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

cmd = '/Users/jaraco/code/pypa/distutils/.tox/py/bin/python', search_path = 1, verbose = 0, dry_run = 0
env = {'COLUMNS': '152', 'HOME': '/Users/jaraco', 'LANG': 'en_US.UTF-8', 'LINES': '53', ...}

    def spawn(cmd, search_path=1, verbose=0, dry_run=0, env=None):  # noqa: C901
        """Run another program, specified as a command list 'cmd', in a new process.
    
        'cmd' is just the argument list for the new process, ie.
        cmd[0] is the program to run and cmd[1:] are the rest of its arguments.
        There is no way to run a program with a name different from that of its
        executable.
    
        If 'search_path' is true (the default), the system's executable
        search path will be used to find the program; otherwise, cmd[0]
        must be the exact path to the executable.  If 'dry_run' is true,
        the command will not actually be run.
    
        Raise DistutilsExecError if running the program fails in any way; just
        return on success.
        """
        # cmd is documented as a list, but just in case some code passes a tuple
        # in, protect our %-formatting code against horrible death
        cmd = list(cmd)
    
        log.info(subprocess.list2cmdline(cmd))
        if dry_run:
            return
    
        if search_path:
            executable = find_executable(cmd[0])
            if executable is not None:
                cmd[0] = executable
    
        env = env if env is not None else dict(os.environ)
    
        if sys.platform == 'darwin':
            from distutils.util import MACOSX_VERSION_VAR, get_macosx_target_ver
    
            macosx_target_ver = get_macosx_target_ver()
            if macosx_target_ver:
                env[MACOSX_VERSION_VAR] = macosx_target_ver
    
        try:
            proc = subprocess.Popen(cmd, env=env)
            proc.wait()
            exitcode = proc.returncode
        except OSError as exc:
            if not DEBUG:
                cmd = cmd[0]
            raise DistutilsExecError(
                "command {!r} failed: {}".format(cmd, exc.args[-1])
            ) from exc
    
        if exitcode:
            if not DEBUG:
                cmd = cmd[0]
>           raise DistutilsExecError(
                "command {!r} failed with exit code {}".format(cmd, exitcode)
            )
E           distutils.errors.DistutilsExecError: command '/Users/jaraco/code/pypa/distutils/.tox/py/bin/python' failed with exit code 1

/Users/jaraco/code/pypa/distutils/distutils/spawn.py:70: DistutilsExecError
----------------------------------------------------------------- Captured stderr call -----------------------------------------------------------------
Traceback (most recent call last):
  File "/var/folders/sx/n5gkrgfx6zd91ymxr2sr9wvw00n8zm/T/tmpznijdwll.py", line 1, in <module>
    from distutils.util import byte_compile
ModuleNotFoundError: No module named 'distutils'
------------------------------------------------------------------ Captured log call -------------------------------------------------------------------
INFO     root:dir_util.py:71 creating here
INFO     root:file_util.py:137 copying boiledeggs.py -> here
INFO     root:util.py:423 writing byte-compilation script '/var/folders/sx/n5gkrgfx6zd91ymxr2sr9wvw00n8zm/T/tmpznijdwll.py'
INFO     root:spawn.py:38 /Users/jaraco/code/pypa/distutils/.tox/py/bin/python /var/folders/sx/n5gkrgfx6zd91ymxr2sr9wvw00n8zm/T/tmpznijdwll.py

[jaraco]

Investigating the second failure mode, I see that "import distutils" is failing in a subprocess when os.getcwd() is not the project directory (and thus this copy of distutils is not on the path). It seems the only reason these tests weren't failing on older Pythons is because distutils was present in the stdlib.

[jaraco]

I see that distutils is "editable-installed" into the tox env, so why on earth is it not finding the package?

I tried running pyproject-build . and it produced a wheel with no distutils package.

In the output for that command, I see:

setuptools_scm._file_finders.git listing git files failed - pretending there aren't any

Hmmm. That shouldn't be the case, considering I have a proper git checkout of the repo.

[jaraco]

After investigating that issue upstream, I confirmed that the git file finders are working as expected, but the problem is that the package finder exclusion, while trying to exclude "dist/" ends up excluding "dist", causing "distutils" to be excluded:

distutils/setup.cfg

Lines 21 to 26 in 16769f8

	[options.packages.find]
	exclude =
	build*
	dist*
	docs*
	tests*

I'd chosen to use dist* because that's the only syntax that seems to exclude dist/*. Or, at least docs* is the only thing that seems to exclude docs/*, and I followed that pattern.

Removing the dist* from that list allows the distutils package to be included in the sdist and fixes the issues with distutils being missing from the virtualenv.

[jaraco]

It does seem as if maybe dist* and build* are unnecessary excludes. Even if those entries are omitted and if a build and dist directory exists, they don't appear in the wheel, probably because they're not tracked in git.