Python 3.12 build error "No module named 'distutils'" #28695

MikeMcC399 · 2024-01-11T15:31:15Z

Current behavior

Executing yarn to build Cypress from source causes build errors when using the current version of Python 3.12.1. The following npm modules are affected:

The error messages include the following line caused by a breaking change in Python 3.12 which is incompatible with the version of node-gyp currently used by Cypress:

ModuleNotFoundError: No module named 'distutils'

Desired behavior

CONTRIBUTING > Requirements should correctly list working prerequisites. The current text says:

python (since we use node-gyp. See their repo for Python version requirements.)
- Note for Debian-based systems: python is pre-installed.
  sudo apt install g++ make cmake meets the additional requirements to run node-gyp in the context of building Cypress from source.

The text should be changed to state that Python 3.11 is the highest version compatible with Cypress install / build from source.

Test code to reproduce

See https://github.com/MikeMcC399/cypress/blob/prime/.github/workflows/build-test.yml

Cypress Version

13.6.2 develop branch 9e1f1e7

Node version

18.15.0

Operating System

Ubuntu 22.04 and Windows 2022

Debug Logs

[5/6] Building fresh packages...
warning Error running install script for optional dependency: "/home/runner/work/cypress/cypress/node_modules/@cypress/get-windows-proxy/node_modules/registry-js: Command failed.
Exit code: 1
Command: prebuild-install || node-gyp rebuild
Arguments:
Directory: /home/runner/work/cypress/cypress/node_modules/@cypress/get-windows-proxy/node_modules/registry-js
info This module is OPTIONAL, you can safely ignore this error
Output:
prebuild-install WARN install No prebuilt binaries found (target=18.15.0 runtime=node arch=x64 libc= platform=linux)
Traceback (most recent call last):
  File \"/home/runner/work/cypress/cypress/node_modules/@npmcli/run-script/node_modules/node-gyp/gyp/gyp_main.py\", line 42, in <module>
    import gyp  # noqa: E402
    ^^^^^^^^^^
  File \"/home/runner/work/cypress/cypress/node_modules/@npmcli/run-script/node_modules/node-gyp/gyp/pylib/gyp/__init__.py\", line 9, in <module>
    import gyp.input
  File \"/home/runner/work/cypress/cypress/node_modules/@npmcli/run-script/node_modules/node-gyp/gyp/pylib/gyp/input.py\", line 19, in <module>
    from distutils.version import StrictVersion
ModuleNotFoundError: No module named 'distutils'
gyp ERR! configure error
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (/home/runner/work/cypress/cypress/node_modules/@npmcli/run-script/node_modules/node-gyp/lib/configure.js:259:16)
gyp ERR! stack     at ChildProcess.emit (node:events:513:28)
gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:291:12)
gyp ERR! System Linux 6.2.0-1018-azure
gyp ERR! command \"/opt/hostedtoolcache/node/18.15.0/x64/bin/node\" \"/home/runner/work/cypress/cypress/node_modules/.bin/node-gyp\" \"rebuild\"
gyp ERR! cwd /home/runner/work/cypress/cypress/node_modules/@cypress/get-windows-proxy/node_modules/registry-js
gyp ERR! node -v v18.15.0
gyp ERR! node-gyp -v v8.4.1
gyp ERR! not ok"
warning Error running install script for optional dependency: "/home/runner/work/cypress/cypress/node_modules/registry-js: Command failed.
Exit code: 1
info This module is OPTIONAL, you can safely ignore this error
Command: prebuild-install || node-gyp rebuild
Arguments:
Directory: /home/runner/work/cypress/cypress/node_modules/registry-js
Output:
prebuild-install WARN install No prebuilt binaries found (target=3 runtime=napi arch=x64 libc= platform=linux)
Traceback (most recent call last):
  File \"/home/runner/work/cypress/cypress/node_modules/@npmcli/run-script/node_modules/node-gyp/gyp/gyp_main.py\", line 42, in <module>
    import gyp  # noqa: E402
    ^^^^^^^^^^
  File \"/home/runner/work/cypress/cypress/node_modules/@npmcli/run-script/node_modules/node-gyp/gyp/pylib/gyp/__init__.py\", line 9, in <module>
    import gyp.input
  File \"/home/runner/work/cypress/cypress/node_modules/@npmcli/run-script/node_modules/node-gyp/gyp/pylib/gyp/input.py\", line 19, in <module>
    from distutils.version import StrictVersion
ModuleNotFoundError: No module named 'distutils'
gyp ERR! configure error
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (/home/runner/work/cypress/cypress/node_modules/@npmcli/run-script/node_modules/node-gyp/lib/configure.js:259:16)
gyp ERR! stack     at ChildProcess.emit (node:events:513:28)
gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:291:12)
gyp ERR! System Linux 6.2.0-1018-azure
gyp ERR! command \"/opt/hostedtoolcache/node/18.15.0/x64/bin/node\" \"/home/runner/work/cypress/cypress/node_modules/.bin/node-gyp\" \"rebuild\"
gyp ERR! cwd /home/runner/work/cypress/cypress/node_modules/registry-js
gyp ERR! node -v v18.15.0
gyp ERR! node-gyp -v v8.4.1
gyp ERR! not ok"
warning Error running install script for optional dependency: "/home/runner/work/cypress/cypress/node_modules/cpu-features: Command failed.
info This module is OPTIONAL, you can safely ignore this error
Exit code: 1
Command: node-gyp rebuild
Arguments:
Directory: /home/runner/work/cypress/cypress/node_modules/cpu-features
Output:
Traceback (most recent call last):
  File \"/home/runner/work/cypress/cypress/node_modules/@npmcli/run-script/node_modules/node-gyp/gyp/gyp_main.py\", line 42, in <module>
    import gyp  # noqa: E402
    ^^^^^^^^^^
  File \"/home/runner/work/cypress/cypress/node_modules/@npmcli/run-script/node_modules/node-gyp/gyp/pylib/gyp/__init__.py\", line 9, in <module>
    import gyp.input
  File \"/home/runner/work/cypress/cypress/node_modules/@npmcli/run-script/node_modules/node-gyp/gyp/pylib/gyp/input.py\", line 19, in <module>
    from distutils.version import StrictVersion
ModuleNotFoundError: No module named 'distutils'
gyp ERR! configure error
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (/home/runner/work/cypress/cypress/node_modules/@npmcli/run-script/node_modules/node-gyp/lib/configure.js:259:16)
gyp ERR! stack     at ChildProcess.emit (node:events:513:28)
gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:291:12)
gyp ERR! System Linux 6.2.0-1018-azure
gyp ERR! command \"/opt/hostedtoolcache/node/18.15.0/x64/bin/node\" \"/home/runner/work/cypress/cypress/node_modules/.bin/node-gyp\" \"rebuild\"
gyp ERR! cwd /home/runner/work/cypress/cypress/node_modules/cpu-features
gyp ERR! node -v v18.15.0
gyp ERR! node-gyp -v v8.4.1
gyp ERR! not ok"

Other

Python version overview

According to https://endoflife.date/python Python 3.11 is supported until Oct 24, 2027, so there should not be any urgent need for Cypress to support Python 3.12 and later.

Defaults

OS \ Distribution	native	GitHub runner
Ubuntu `22.04`	`3.10.12`	`3.10.12`
Windows 2022	-	`3.9.13`
Windows 11	-	N/A

Deprecations

Python 3.12 introduces a breaking change:

PEP 632: Remove the distutils package. See the migration guide for advice replacing the APIs it provided. The third-party Setuptools package continues to provide distutils, if you still require it in Python 3.12 and beyond.

Support for Python 3.12 was added to node-gyp@10.0.0 (see issue nodejs/node-gyp#2869) in gyp@0.16.1 taking care of the breaking change in Python. node-gyp@10.0.0 is however not (yet) used by Cypress, meaning that Python 3.12 cannot yet be used cleanly either.

The text was updated successfully, but these errors were encountered:

MikeMcC399 · 2024-01-11T15:34:05Z

I can offer to submit a PR to update the text in CONTRIBUTING > Requirements accordingly.

There is also a related issue that the version of node-gyp used by Cypress is not compatible with the Windows 11 SDK. For clarity I will submit that as a separate issue.

Edit: See also node-gyp build error "missing any Windows SDK" #28703

jennifer-shehane · 2024-01-11T16:18:43Z

@MikeMcC399 Thanks, that does seem like the most reasonable PR to start.

MikeMcC399 · 2024-01-12T11:41:34Z

@jennifer-shehane

The Python default on Ubuntu is version 3.10, which works fine. It's only Windows where a user might install Python 3.12 (latest) and run into issues, however since I couldn't find a solution for node-gyp build error "missing any Windows SDK" #28703 so far, it won't be very helpful to add the Python version restriction if it still doesn't work. I'll come back to this issue if I find something more complete and usable which can be documented.

MikeMcC399 · 2024-01-20T10:42:22Z

Issue node-gyp build error "missing any Windows SDK" #28703 is resolved through PRs chore(deps): refresh dockerode@3.3.1 #28726 and docs: Install Windows 10 SDK for yarn install #28760
This issue is planned to be resolved through the documentation PR docs: Python 3.11 is highest compatible version for yarn install #28759

MikeMcC399 · 2024-01-20T11:02:44Z

Comment: According to https://endoflife.date/python Python 3.11 is supported until Oct 24, 2027, so there should not be any urgent need for Cypress to support Python 3.12 and later.

jennifer-shehane added the process: contributing Related to contributing to the Cypress codebase label Jan 11, 2024

MikeMcC399 changed the title ~~Python 3.12 build error "No module named 'distutils"~~ Python 3.12 build error "No module named 'distutils'" Jan 11, 2024

MikeMcC399 mentioned this issue Jan 11, 2024

node-gyp build error "missing any Windows SDK" #28703

Closed

MikeMcC399 mentioned this issue Jan 20, 2024

docs: Python 3.11 is highest compatible version for yarn install #28759

Merged

jennifer-shehane closed this as completed in #28759 Jan 22, 2024

MikeMcC399 mentioned this issue Apr 11, 2024

docs: add contributor instructions for Python 3.11 on Ubuntu 24.04 #29311

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Python 3.12 build error "No module named 'distutils'" #28695

Python 3.12 build error "No module named 'distutils'" #28695

MikeMcC399 commented Jan 11, 2024 •

edited

MikeMcC399 commented Jan 11, 2024 •

edited

jennifer-shehane commented Jan 11, 2024

MikeMcC399 commented Jan 12, 2024 •

edited

MikeMcC399 commented Jan 20, 2024

MikeMcC399 commented Jan 20, 2024

Python 3.12 build error "No module named 'distutils'" #28695

Python 3.12 build error "No module named 'distutils'" #28695

Comments

MikeMcC399 commented Jan 11, 2024 • edited

Current behavior

Desired behavior

Test code to reproduce

Cypress Version

Node version

Operating System

Debug Logs

Other

Defaults

Deprecations

MikeMcC399 commented Jan 11, 2024 • edited

jennifer-shehane commented Jan 11, 2024

MikeMcC399 commented Jan 12, 2024 • edited

MikeMcC399 commented Jan 20, 2024

MikeMcC399 commented Jan 20, 2024

MikeMcC399 commented Jan 11, 2024 •

edited

MikeMcC399 commented Jan 11, 2024 •

edited

MikeMcC399 commented Jan 12, 2024 •

edited