PermissionError on Windows runner for CMake FetchContent dependency for Python 3.7

[golmschenk]

Hello! I'm running into an issue where, for a Windows CI runner with Python 3.7, the build fails due to a PermissionError: [WinError 5] Access is denied. It appears to be occurring due to a CMake FetchContent dependency from a git repository. It seems the build is not properly releasing the git related resources during the wheel build. The build works fine for both macOS and Linux. The build works fine for Windows with Python 3.8.

This issue was originally posted to pypa/cibuildwheel#1523 before being moved here. There, @henryiii noted that the issue is related to the implementation of TemporaryDirectory and shutil.rmtree in Python 3.7 which was then fixed in Python 3.8.

I setup a small example repository demonstrating this issue here: https://github.com/golmschenk/cibuildwheel_fetchcontent_permission_error_example.
The failing build using Python 3.7 can be seen here: https://github.com/golmschenk/cibuildwheel_fetchcontent_permission_error_example/actions/runs/5270068999/jobs/9529084594.
The successful build using Python 3.8 can be seen here:
https://github.com/golmschenk/cibuildwheel_fetchcontent_permission_error_example/actions/runs/5290893595/jobs/9575683760

The error appears as:

    *** Making wheel...
    Traceback (most recent call last):
      File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-4pku3vah\cp37-win32\build\venv\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
        main()
      File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-4pku3vah\cp37-win32\build\venv\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
        json_out['return_val'] = hook(**hook_input['kwargs'])
      File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-4pku3vah\cp37-win32\build\venv\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 252, in build_wheel
        metadata_directory)
      File "C:\Users\RUNNER~1\AppData\Local\Temp\pip-build-env-ss4tjy2i\overlay\Lib\site-packages\scikit_build_core\build\__init__.py", line 36, in build_wheel
        editable=False,
      File "C:\Users\RUNNER~1\AppData\Local\Temp\pip-build-env-ss4tjy2i\overlay\Lib\site-packages\scikit_build_core\build\wheel.py", line 288, in _build_wheel_impl
        f"import _{normalized_name}_editable\n".encode(),
      File "C:\Users\runneradmin\AppData\Local\pypa\cibuildwheel\Cache\nuget-cpython\pythonx86.3.7.9\tools\lib\tempfile.py", line 807, in __exit__
        self.cleanup()
      File "C:\Users\runneradmin\AppData\Local\pypa\cibuildwheel\Cache\nuget-cpython\pythonx86.3.7.9\tools\lib\tempfile.py", line 811, in cleanup
        _shutil.rmtree(self.name)
      File "C:\Users\runneradmin\AppData\Local\pypa\cibuildwheel\Cache\nuget-cpython\pythonx86.3.7.9\tools\lib\shutil.py", line 516, in rmtree
        return _rmtree_unsafe(path, onerror)
      File "C:\Users\runneradmin\AppData\Local\pypa\cibuildwheel\Cache\nuget-cpython\pythonx86.3.7.9\tools\lib\shutil.py", line 395, in _rmtree_unsafe
        _rmtree_unsafe(fullname, onerror)
      File "C:\Users\runneradmin\AppData\Local\pypa\cibuildwheel\Cache\nuget-cpython\pythonx86.3.7.9\tools\lib\shutil.py", line 395, in _rmtree_unsafe
        _rmtree_unsafe(fullname, onerror)
      File "C:\Users\runneradmin\AppData\Local\pypa\cibuildwheel\Cache\nuget-cpython\pythonx86.3.7.9\tools\lib\shutil.py", line 395, in _rmtree_unsafe
        _rmtree_unsafe(fullname, onerror)
      [Previous line repeated 3 more times]
      File "C:\Users\runneradmin\AppData\Local\pypa\cibuildwheel\Cache\nuget-cpython\pythonx86.3.7.9\tools\lib\shutil.py", line 400, in _rmtree_unsafe
        onerror(os.unlink, fullname, sys.exc_info())
      File "C:\Users\runneradmin\AppData\Local\pypa\cibuildwheel\Cache\nuget-cpython\pythonx86.3.7.9\tools\lib\shutil.py", line 398, in _rmtree_unsafe
        os.unlink(fullname)
    PermissionError: [WinError 5] Access is denied: 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\tmpwvkc37iy\\build\\_deps\\fortran_stdlib-src\\.git\\objects\\pack\\pack-faae3e6a7f3ee7b73a650fc640815b9d8dbf92df.idx'

Thank you for your time!

[LecrisUT]

Note that Python 3.7 is EOL at the end of this month. It might be prudent to increase the minimum Python version requirement.

[henryiii]

We'll keep 3.7 for a bit; dropping a Python version for a stable package is a great idea, but for one that's still being developed, it can have bad consequences. Once most of our users drop 3.7, we can to, or once we feel we have enough features that dropping won't hurt users who still want to use 3.7. Plus it's not EoL yet! :)

So I'll need to work around this, but shouldn't be too hard.

[JeanChristopheMorinPerso]

My communities still heavily rely on 3.7, so it would be nice to keep support for a while :) And from what I know, we don't gain much from dropping support for 3.7 right?

[henryiii]

Not much, no, just the ability to not write this workaround. :P

(Nothing like avoiding or dropping 3.6, like annotations, etc)

New designs get to use positional-only arguments, and do slightly less back porting. See https://iscinumpy.dev/post/python-38/.

[henryiii]

IMO, the next major drop will be when we can drop 3.9. 3.10 is a big deal, like 3.6 and 3.7 were. That's why repo-review requires 3.10+. :)