Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[3.6.x] Unable to install dependencies w/ Python 3.11 on M1 #15394

Closed
dlqqq opened this issue Nov 12, 2023 · 1 comment
Closed

[3.6.x] Unable to install dependencies w/ Python 3.11 on M1 #15394

dlqqq opened this issue Nov 12, 2023 · 1 comment
Labels
bug os:macos Issues specific to macOS operating system tag:Build System
Milestone
3.6.x

Comments

@dlqqq
Copy link
Contributor

dlqqq commented Nov 12, 2023 •
edited

Search keyword: invalid mode: 'rU' while trying to load binding.gyp

Description

I'm unable to install dependencies w/ Python 3.11 on an M1 Mac (Darwin ARM64).

This results in an error:

jlpm output 
% jlpm
yarn install v1.21.1
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
warning Pattern ["node-fetch@cjs"] is trying to unpack in the same destination "/Users/qiu/Library/Caches/Yarn/v6/npm-node-fetch-2.6.7-24de9fba827e3b4ae44dc8b20256a379160052ad-integrity/node_modules/node-fetch" as pattern ["node-fetch@^2.6.1","node-fetch@^2.6.1","node-fetch@^2.6.0","node-fetch@^2.6.0","node-fetch@^2.6.0","node-fetch@^2.6.0","node-fetch@^2.6.1","node-fetch@^2.6.0","node-fetch@^2.6.1"]. This could result in non-deterministic behavior, skipping.
[3/4] 🔗  Linking dependencies...
warning "@typescript-eslint/eslint-plugin > tsutils@3.21.0" has unmet peer dependency "typescript@>=2.8.0 || >= 3.2.0-dev || >= 3.3.0-dev || >= 3.4.0-dev || >= 3.5.0-dev || >= 3.6.0-dev || >= 3.6.0-beta || >= 3.7.0-dev || >= 3.7.0-beta".
warning " > typedoc@0.21.2" has unmet peer dependency "typescript@4.0.x || 4.1.x || 4.2.x || 4.3.x".
warning " > @jupyterlab/ui-components@3.6.6" has unmet peer dependency "react@^17.0.1".
warning "workspace-aggregator-a1db0ff0-bb36-46b8-8453-f8cd9096f8fc > @jupyterlab/example-federated-middle > @lumino/coreutils@1.12.1" has unmet peer dependency "crypto@1.0.1".
warning "workspace-aggregator-a1db0ff0-bb36-46b8-8453-f8cd9096f8fc > @jupyterlab/codemirror > y-codemirror@3.0.1" has unmet peer dependency "yjs@^13.5.17".
warning "workspace-aggregator-a1db0ff0-bb36-46b8-8453-f8cd9096f8fc > @jupyterlab/extensionmanager > react-paginate@6.5.0" has incorrect peer dependency "react@^16.0.0".
warning "workspace-aggregator-a1db0ff0-bb36-46b8-8453-f8cd9096f8fc > @jupyterlab/json-extension > react-highlighter@0.4.3" has incorrect peer dependency "react@^0.14.0 || ^15.0.0 || ^16.0.0".
warning "workspace-aggregator-a1db0ff0-bb36-46b8-8453-f8cd9096f8fc > @jupyterlab/ui-components > babel-loader@8.2.2" has unmet peer dependency "webpack@>=2".
warning "workspace-aggregator-a1db0ff0-bb36-46b8-8453-f8cd9096f8fc > @jupyterlab/vdom > @nteract/transform-vdom@4.0.16-alpha.0" has incorrect peer dependency "react@^16.3.2".
warning "workspace-aggregator-a1db0ff0-bb36-46b8-8453-f8cd9096f8fc > @jupyterlab/galata > svgo-loader@2.2.2" has unmet peer dependency "svgo@^1.0.0".
warning "workspace-aggregator-a1db0ff0-bb36-46b8-8453-f8cd9096f8fc > @jupyterlab/buildutils > verdaccio > clipanion@3.2.0-rc.14" has unmet peer dependency "typanion@*".
warning "workspace-aggregator-a1db0ff0-bb36-46b8-8453-f8cd9096f8fc > @jupyterlab/ui-components > @storybook/addon-actions > @storybook/api > @reach/router@1.3.4" has incorrect peer dependency "react-dom@15.x || 16.x || 16.4.0-alpha.0911da3".
warning Workspaces can only be enabled in private projects.
warning Workspaces can only be enabled in private projects.
[4/4] 🔨  Building fresh packages...
[1/5] ⡀ @fortawesome/fontawesome-free
[-/5] ⡀ waiting...
[-/5] ⡀ waiting...
[4/5] ⡀ canvas
error /Users/qiu/Repositories/jupyterlab/node_modules/canvas: Command failed.
Exit code: 1
Command: node-pre-gyp install --fallback-to-build --update-binary
Arguments:
Directory: /Users/qiu/Repositories/jupyterlab/node_modules/canvas
Output:
node-pre-gyp info it worked if it ends with ok
node-pre-gyp info using node-pre-gyp@1.0.5
node-pre-gyp info using node@18.16.0 | darwin | arm64
node-pre-gyp http GET https://github.com/Automattic/node-canvas/releases/download/v2.11.0/canvas-v2.11.0-node-v108-darwin-unknown-arm64.tar.gz
node-pre-gyp ERR! install response status 404 Not Found on https://github.com/Automattic/node-canvas/releases/download/v2.11.0/canvas-v2.11.0-node-v108-darwin-unknown-arm64.tar.gz
node-pre-gyp WARN Pre-built binaries not installable for canvas@2.11.0 and node@18.16.0 (node-v108 ABI, unknown) (falling back to source compile with node-gyp)
node-pre-gyp WARN Hit error response status 404 Not Found on https://github.com/Automattic/node-canvas/releases/download/v2.11.0/canvas-v2.11.0-node-v108-darwin-unknown-arm64.tar.gz
gyp info it worked if it ends with ok
gyp info using node-gyp@5.1.1
gyp info using node@18.16.0 | darwin | arm64
gyp info ok
gyp info it worked if it ends with ok
gyp info using node-gyp@5.1.1
gyp info using node@18.16.0 | darwin | arm64
gyp info find Python using Python version 3.11.5 found at "/Users/qiu/miniconda3/envs/jl3-dev/bin/python"
(node:80286) [DEP0150] DeprecationWarning: Setting process.config is deprecated. In the future the property will be read-only.
(Use `node --trace-deprecation ...` to show where the warning was created)
gyp info spawn /Users/qiu/miniconda3/envs/jl3-dev/bin/python
gyp info spawn args [
gyp info spawn args   '/Users/qiu/Repositories/jupyterlab/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/qiu/Repositories/jupyterlab/node_modules/canvas/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/qiu/Repositories/jupyterlab/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/qiu/Library/Caches/node-gyp/18.16.0/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/Users/qiu/Library/Caches/node-gyp/18.16.0',
gyp info spawn args   '-Dnode_gyp_dir=/Users/qiu/Repositories/jupyterlab/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/Users/qiu/Library/Caches/node-gyp/18.16.0/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/Users/qiu/Repositories/jupyterlab/node_modules/canvas',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
Traceback (most recent call last):
  File "/Users/qiu/Repositories/jupyterlab/node_modules/node-gyp/gyp/gyp_main.py", line 50, in 
    sys.exit(gyp.script_main())
             ^^^^^^^^^^^^^^^^^
  File "/Users/qiu/Repositories/jupyterlab/node_modules/node-gyp/gyp/pylib/gyp/__init__.py", line 554, in script_main
    return main(sys.argv[1:])
           ^^^^^^^^^^^^^^^^^^
  File "/Users/qiu/Repositories/jupyterlab/node_modules/node-gyp/gyp/pylib/gyp/__init__.py", line 547, in main
    return gyp_main(args)
           ^^^^^^^^^^^^^^
  File "/Users/qiu/Repositories/jupyterlab/node_modules/node-gyp/gyp/pylib/gyp/__init__.py", line 520, in gyp_main
    [generator, flat_list, targets, data] = Load(
                                            ^^^^^
  File "/Users/qiu/Repositories/jupyterlab/node_modules/node-gyp/gyp/pylib/gyp/__init__.py", line 136, in Load
    result = gyp.input.Load(build_files, default_variables, includes[:],
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/qiu/Repositories/jupyterlab/node_modules/node-gyp/gyp/pylib/gyp/input.py", line 2782, in Load
    LoadTargetBuildFile(build_file, data, aux_data,
  File "/Users/qiu/Repositories/jupyterlab/node_modules/node-gyp/gyp/pylib/gyp/input.py", line 391, in LoadTargetBuildFile
    build_file_data = LoadOneBuildFile(build_file_path, data, aux_data,
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/qiu/Repositories/jupyterlab/node_modules/node-gyp/gyp/pylib/gyp/input.py", line 234, in LoadOneBuildFile
    build_file_contents = open(build_file_path, 'rU').read()
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: invalid mode: 'rU' while trying to load binding.gyp
gyp ERR! configure error
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (/Users/qiu/Repositories/jupyterlab/node_modules/node-gyp/lib/configure.js:351:16)
gyp ERR! stack     at ChildProcess.emit (node:events:513:28)
gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:291:12)
gyp ERR! System Darwin 22.6.0
gyp ERR! command "/Users/qiu/miniconda3/envs/jl3-dev/bin/node" "/Users/qiu/Repositories/jupyterlab/node_modules/node-gyp/bin/node-gyp.js" "configure" "--fallback-to-build" "--update-binary" "--module=/Users/qiu/Repositories/jupyterlab/node_modules/canvas/build/Release/canvas.node" "--module_name=canvas" "--module_path=/Users/qiu/Repositories/jupyterlab/node_modules/canvas/build/Release" "--napi_version=8" "--node_abi_napi=napi" "--napi_build_version=0" "--node_napi_label=node-v108"
gyp ERR! cwd /Users/qiu/Repositories/jupyterlab/node_modules/canvas
gyp ERR! node -v v18.16.0
gyp ERR! node-gyp -v v5.1.1
gyp ERR! not ok
node-pre-gyp ERR! build error
node-pre-gyp ERR! stack Error: Failed to execute '/Users/qiu/miniconda3/envs/jl3-dev/bin/node /Users/qiu/Repositories/jupyterlab/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --update-binary --module=/Users/qiu/Repositories/jupyterlab/node_modules/canvas/build/Release/canvas.node --module_name=canvas --module_path=/Users/qiu/Repositories/jupyterlab/node_modules/canvas/build/Release --napi_version=8 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v108' (1)
node-pre-gyp ERR! stack     at ChildProcess. (/Users/qiu/Repositories/jupyterlab/node_modules/@mapbox/node-pre-gyp/lib/util/compile.js:89:23)
node-pre-gyp ERR! stack     at ChildProcess.emit (node:events:513:28)
node-pre-gyp ERR! stack     at maybeClose (node:internal/child_process:1091:16)
node-pre-gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:302:5)
node-pre-gyp ERR! System Darwin 22.6.0
node-pre-gyp ERR! command "/Users/qiu/miniconda3/envs/jl3-dev/bin/node" "/Users/qiu/Repositories/jupyterlab/node_modules/canvas/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build" "--update-binary"
node-pre-gyp ERR! cwd /Users/qiu/Repositories/jupyterlab/node_modules/canvas
node-pre-gyp ERR! node -v v18.16.0
node-pre-gyp ERR! node-pre-gyp -v v1.0.5

This happens because canvas does not ship Darwin ARM64 prebuilds, and therefore must be built by node-gyp. However, node-gyp is being resolved to 7.1.2 in yarn.lock, which does not support Python 3.11. The oldest supported version of node-gyp is currently 9.0.0.

Reproduce

On any platform with no canvas prebuilds, including M1 Mac:

# cwd := jupyterlab repo top
git checkout 3.6.x && git pull
conda create -yn jl3-dev python=3.11
conda activate jl3-dev

# install node via Conda
conda install -c conda-forge nodejs
# returns v18.16.0
node --version

# see: https://jupyterlab.readthedocs.io/en/stable/developer/contributing.html#installing-node-js-and-jlpm
# see also: https://github.com/jupyterlab/jupyterlab/issues/13938
conda install -c conda-forge pkg-config pango libpng cairo jpeg giflib librsvg glib pixman
export PKG_CONFIG_PATH=$CONDA_PREFIX/lib/pkgconfig

# this should produce the error
# you can get it to show again via `jlpm install`
pip install -e ".[dev,test]"

Expected behavior

I should be able to install the dependencies for the 3.6.x branch.
@dlqqq dlqqq added the bug label Nov 12, 2023
@jupyterlab-probot jupyterlab-probot bot added the status:Needs Triage Applied to new issues that need triage label Nov 12, 2023
@dlqqq dlqqq mentioned this issue Nov 12, 2023
Merged
@JasonWeill JasonWeill added this to the 3.6.x milestone Nov 14, 2023
@JasonWeill JasonWeill added tag:Build System os:macos Issues specific to macOS operating system and removed status:Needs Triage Applied to new issues that need triage labels Nov 14, 2023
@JasonWeill JasonWeill mentioned this issue Nov 14, 2023
Closed
@dlqqq
Copy link
Contributor Author

dlqqq commented Nov 17, 2023

Fixed by #15395.

@dlqqq dlqqq closed this as completed Nov 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug os:macos Issues specific to macOS operating system tag:Build System
Projects
None yet
Milestone
3.6.x
Development

No branches or pull requests
2 participants
@dlqqq @JasonWeill