Mismatched x64 and ARM64 architectures on M1 Mac

[longzheng]

I had installed sharp 0.27.1 earlier today on my M1 Mac and used it without problems for several hours, resizing and converting various images.

Then a few hours ago I was changing some yarn packages and sharp stopped working all of a sudden, and I was getting this message whenever I ran a node script with require("sharp")

$ node processPhotos.js
/Users/longzheng/Documents/GitHub/photos/node_modules/sharp/lib/constructor.js:34
  throw new Error(error);
  ^

Error: 
Something went wrong installing the "sharp" module

dlopen(/Users/longzheng/Documents/GitHub/photos/node_modules/sharp/build/Release/sharp.node, 1): Symbol not found: __ZTVN4vips7VOptionE
  Referenced from: /Users/longzheng/Documents/GitHub/photos/node_modules/sharp/build/Release/sharp.node
  Expected in: flat namespace
 in /Users/longzheng/Documents/GitHub/photos/node_modules/sharp/build/Release/sharp.node

- Remove the "node_modules/sharp" directory then run
  "npm install --ignore-scripts=false --verbose" and look for errors
- Consult the installation documentation at https://sharp.pixelplumbing.com/install
- Search for this error at https://github.com/lovell/sharp/issues

    at Object.<anonymous> (/Users/longzheng/Documents/GitHub/photos/node_modules/sharp/lib/constructor.js:34:9)
    at Module._compile (node:internal/modules/cjs/loader:1091:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1120:10)
    at Module.load (node:internal/modules/cjs/loader:971:32)
    at Function.Module._load (node:internal/modules/cjs/loader:812:14)
    at Module.require (node:internal/modules/cjs/loader:995:19)
    at require (node:internal/modules/cjs/helpers:92:18)
    at Object.<anonymous> (/Users/longzheng/Documents/GitHub/photos/node_modules/sharp/lib/index.js:3:15)
    at Module._compile (node:internal/modules/cjs/loader:1091:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1120:10)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

I've tried various troubleshooting steps without any effect

• reinstalling vips with brew reinstall vips
• reinstalling sharp with yarn add --force sharp
• clearing yarn cache with yarn cache clear
• deleting node_modules and reinstalling packages with yarn install
• restarting macOS

My env

% npx envinfo --binaries --system

  System:
    OS: macOS 11.2.1
    CPU: (8) x64 Apple M1
    Memory: 37.70 MB / 16.00 GB
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 15.9.0 - /usr/local/bin/node
    Yarn: 1.22.5 - ~/.yarn/bin/yarn
    npm: 7.5.3 - /usr/local/bin/npm

I'm completely dumbfounded why it was working well then suddenly stopped working.

[lovell]

I was changing some yarn packages

What's the output of yarn list --pattern sharp ?

See also #2460 (comment)

[longzheng]

I was changing some yarn packages

What's the output of yarn list --pattern sharp ?

See also #2460 (comment)

% yarn list --pattern sharp
yarn list v1.22.5
└─ sharp@0.27.1
✨  Done in 0.85s.

Yeah I did see #2460 however it's weird because it was working all fine for me for several hours. I guess I'll try some other suggestions in that thread.

[longzheng]

Logs from running npm run install in node_modules/sharp

longzheng@Longs-MBP sharp % npm run install

> sharp@0.27.1 install
> (node install/libvips && node install/dll-copy && prebuild-install) || (node-gyp rebuild && node install/dll-copy)

info sharp Detected globally-installed libvips v8.10.5
info sharp Building from source via node-gyp
gyp info it worked if it ends with ok
gyp info using node-gyp@7.1.2
gyp info using node@15.9.0 | darwin | x64
gyp info find Python using Python version 3.9.2 found at "/opt/homebrew/opt/python@3.9/bin/python3.9"
gyp info spawn /opt/homebrew/opt/python@3.9/bin/python3.9
gyp info spawn args [
gyp info spawn args   '/Users/longzheng/Documents/GitHub/photos/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/longzheng/Documents/GitHub/photos/node_modules/sharp/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/longzheng/Documents/GitHub/photos/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/longzheng/Library/Caches/node-gyp/15.9.0/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/Users/longzheng/Library/Caches/node-gyp/15.9.0',
gyp info spawn args   '-Dnode_gyp_dir=/Users/longzheng/Documents/GitHub/photos/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/Users/longzheng/Library/Caches/node-gyp/15.9.0/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/Users/longzheng/Documents/GitHub/photos/node_modules/sharp',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
  CC(target) Release/obj.target/nothing/../node-addon-api/nothing.o
  LIBTOOL-STATIC Release/nothing.a
warning: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: archive library: Release/nothing.a the table of contents is empty (no object file members in the library define global symbols)
  TOUCH Release/obj.target/libvips-cpp.stamp
  CXX(target) Release/obj.target/sharp/src/common.o
  CXX(target) Release/obj.target/sharp/src/metadata.o
  CXX(target) Release/obj.target/sharp/src/stats.o
  CXX(target) Release/obj.target/sharp/src/operations.o
  CXX(target) Release/obj.target/sharp/src/pipeline.o
  CXX(target) Release/obj.target/sharp/src/utilities.o
  CXX(target) Release/obj.target/sharp/src/sharp.o
  SOLINK_MODULE(target) Release/sharp.node
ld: warning: ignoring file /opt/homebrew/Cellar/vips/8.10.5_2/lib/libvips-cpp.dylib, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /opt/homebrew/Cellar/vips/8.10.5_2/lib/libvips.dylib, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /opt/homebrew/Cellar/glib/2.66.7/lib/libglib-2.0.dylib, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /opt/homebrew/Cellar/glib/2.66.7/lib/libgobject-2.0.dylib, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /opt/homebrew/opt/gettext/lib/libintl.dylib, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
gyp info ok 
longzheng@Longs-MBP sharp % 

[lovell]

info sharp Detected globally-installed libvips v8.10.5
...
gyp info using node@15.9.0 | darwin | x64
...
ld: warning: ignoring file /opt/homebrew/Cellar/vips/8.10.5_2/lib/libvips-cpp.dylib, building for macOS-x86_64 but attempting to link with file built for macOS-arm64

vips is arm64 but you're using x64 Node.js - the architectures must match.

[longzheng]

That's weird. My node 15.9.0 is returning x64, is it suppose to return arm64?

% node -v
v15.9.0
% node -p 'process.arch'
x64
% arch
arm64

[lovell]

If you're using x64 Node.js on an ARM64 CPU then homebrew-installed ARM64 libraries will not work with it. Use brew remove vips to remove these then re-install sharp - it will detect x64 and use its own x64 binaries.

Alternatively use ARM64 Node.js.

[longzheng]

That's so weird, I don't know how I ended up in this state. 😕 I thought I had been using the arm64 version of vips when it sounds like I wasn't.

brew remove vips
rm -rf node_modules
yarn install

and now sharp works correctly.

Thanks for your advice.

[longzheng]

Maybe if I can leave a suggestion for the installation guide https://sharp.pixelplumbing.com/install#apple-m1 is that people should verify what architecture is their node running in with node -p 'process.arch' before installing brew install vips

[lovell]

Glad you worked it out. There's a discussion later in #2460 (comment) about the mismatched architecture problem - definitely keen to make this easier.

What's the output of running /usr/bin/uname -p at the command line on your M1 machine?

[lovell]

It looks like uname won't help detect emulation but sysctl sysctl.proc_translated will (based on https://stackoverflow.com/a/65347893 ).

[longzheng]

It looks like uname won't help detect emulation but sysctl sysctl.proc_translated will (based on https://stackoverflow.com/a/65347893 ).

Here's some outputs for debugging

% usr/bin/uname -p
zsh: no such file or directory: usr/bin/uname

% sysctl sysctl.proc_translated
sysctl.proc_translated: 0

% arch
arm64

% file $(which node)
/usr/local/bin/node: Mach-O 64-bit executable x86_64

I think the most confusing part was that even though node 15 supports arm64 natively, the default package install still runs in x64 mode.

[lovell]

See #2460 (comment)

[longzheng]

Perfect thank you.

[rama41222]

If you're using x64 Node.js on an ARM64 CPU then homebrew-installed ARM64 libraries will not work with it. Use brew remove vips to remove these then re-install sharp - it will detect x64 and use its own x64 binaries.

Alternatively use ARM64 Node.js.

Worked! Thanks

[thuynt99]

i use node ver 14 and resolve with sharp: "^0.29.3"

[lovell]

For anyone else running into this, please upgrade to the latest version of sharp and this should all "just work" e.g. it will now ignore a Homebrew-installed vips package of the wrong architecture.