parcel fails to install/run properly on MacOS due to OS quaranteeing prebuilt sourcemap binaries

[jenswegar]

🐛 bug report

I followed the steps to install parcel@next (beta.1) according to readme of the project, but even with that minimal example running parcel failed on my system. It seems the prebuilt sourcemap.node binaries are blocked by MacOS and then the install tries to build it's own binaries, which also fails. Not sure if this can be fixed other than by signing the prebuilt binaries via Apple... This might also be related to #4426, but since the commands and error outputs weren't exactly the same on my system I figured a separate report is better.

🎛 Configuration (.babelrc, package.json, cli command)

package.json

{
  "name": "parcel-test",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "dev": "parcel index.html",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "parcel": "^2.0.0-beta.1"
  }
}

command

npm run dev

🤔 Expected Behavior

Parcel should install and run the minimal example without issues.

😯 Current Behavior

parcel fails to run with an error related to sourcemap.node.

> parcel-test@1.0.0 dev /Volumes/CodeProjects/parcel-test
> parcel index.html

internal/modules/cjs/loader.js:1302
  return process.dlopen(module, path.toNamespacedPath(filename));
                 ^

Error: dlopen(/Volumes/CodeProjects/parcel-test/node_modules/@parcel/source-map/build/Release/sourcemap.node, 1): Symbol not found: __ZN4Napi7details31needs_objectwrap_destructor_fixE
  Referenced from: /Volumes/CodeProjects/parcel-test/node_modules/@parcel/source-map/build/Release/sourcemap.node
  Expected in: flat namespace
 in /Volumes/CodeProjects/parcel-test/node_modules/@parcel/source-map/build/Release/sourcemap.node
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:1302:18)
    at Module.load (internal/modules/cjs/loader.js:1100:32)
    at Function.Module._load (internal/modules/cjs/loader.js:962:14)
    at Module.require (internal/modules/cjs/loader.js:1140:19)
    at require (internal/modules/cjs/helpers.js:75:18)
    at load (/Volumes/CodeProjects/parcel-test/node_modules/node-gyp-build/index.js:21:10)
    at Object.<anonymous> (/Volumes/CodeProjects/parcel-test/node_modules/@parcel/source-map/dist/node.js:14:43)
    at Module._compile (internal/modules/cjs/loader.js:1251:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1272:10)
    at Module.load (internal/modules/cjs/loader.js:1100:32)
npm ERR! code ELIFECYCLE
npm ERR! errno 1

Test run output suggests it's trying to use a corrupt version of sourcemap.node that it tried to build itself instead of the prebuilt ones that come with parcel.

Note during install MacOS security prompt pops up saying:

“node.napi.glibc.node” cannot be opened because the developer cannot be verified.

The dialog gives the options to move the binary to the trash or cancel. This happens twice during npm install process.

💁 Possible Solution

A workaround I found is to do the npm install once. The install will create the folder node_modules/@parcel/source-map/build, which apparently is the sourcemap.node that will be used if the prebuilt binaries didn't work. Delete this folder, and run parcel again. MacOS will again complain that the developer cannot be verified for “node.napi.glibc.node”. Cancel the dialog, open System Preferences -> Security and Privacy, and there should be a security prompt about "node.napi.glibc.node" and an option to allow the binary anyway. Click allow. Now run parcel again and it should work (for me it also complains about a watcher.node, but following the same procedure of allowing the binary works there as well).

I suspect this error has to do with MacOS quaranteeing any downloaded binaries by default unless they are signed. During the install, an error occurs that suggests that the node-gyp-build tries to load the prebuilt binaries, but since the code signature fails it falls back to trying to build itself. For some reason this build also fails. Error when the install attempts to load the prebuilt binary is below. Interesting line imo is right at the beginning (Error: dlopen(/V...), but leaving the rest for more context.

npm info lifecycle @parcel/source-map@2.0.0-alpha.4.13~install: @parcel/source-map@2.0.0-alpha.4.13

> @parcel/source-map@2.0.0-alpha.4.13 install /Volumes/CodeProjects/parcel-test/node_modules/@parcel/source-map
> node-gyp-build

internal/modules/cjs/loader.js:1302
  return process.dlopen(module, path.toNamespacedPath(filename));
                 ^

Error: dlopen(/Volumes/CodeProjects/parcel-test/node_modules/@parcel/source-map/prebuilds/darwin-x64/node.napi.glibc.node, 1): no suitable image found.  Did find:
	/Volumes/CodeProjects/parcel-test/node_modules/@parcel/source-map/prebuilds/darwin-x64/node.napi.glibc.node: code signature in (/Volumes/CodeProjects/parcel-test/node_modules/@parcel/source-map/prebuilds/darwin-x64/node.napi.glibc.node) not valid for use in process using Library Validation: library load disallowed by system policy
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:1302:18)
    at Module.load (internal/modules/cjs/loader.js:1100:32)
    at Function.Module._load (internal/modules/cjs/loader.js:962:14)
    at Module.require (internal/modules/cjs/loader.js:1140:19)
    at require (internal/modules/cjs/helpers.js:75:18)
    at load (/Volumes/CodeProjects/parcel-test/node_modules/node-gyp-build/index.js:21:10)
    at Object.<anonymous> (/Volumes/CodeProjects/parcel-test/node_modules/node-gyp-build/build-test.js:19:19)
    at Module._compile (internal/modules/cjs/loader.js:1251:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1272:10)
    at Module.load (internal/modules/cjs/loader.js:1100:32)

gyp info it worked if it ends with ok
gyp verb cli [
gyp verb cli   '/usr/local/Cellar/node/14.8.0/bin/node',
gyp verb cli   '/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js',
gyp verb cli   'rebuild'
gyp verb cli ]
gyp info using node-gyp@5.1.0
gyp info using node@14.8.0 | darwin | x64
gyp verb command rebuild []
gyp verb command clean []
gyp verb clean removing "build" directory
gyp verb command configure []
gyp verb find Python Python is not set from command line or npm configuration
gyp verb find Python Python is not set from environment variable PYTHON
gyp verb find Python checking if "python" can be used
gyp verb find Python - executing "python" to get executable path
gyp verb find Python - executable path is "/System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python"
gyp verb find Python - executing "/System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python" to get version
gyp verb find Python - version is "2.7.16"
gyp info find Python using Python version 2.7.16 found at "/System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python"
gyp verb get node dir no --target version specified, falling back to host node version: 14.8.0
gyp verb command install [ '14.8.0' ]
gyp verb install input version string "14.8.0"
gyp verb install installing version: 14.8.0
gyp verb install --ensure was passed, so won't reinstall if already installed
gyp verb install version is already installed, need to check "installVersion"
gyp verb got "installVersion" 9
gyp verb needs "installVersion" 9
gyp verb install version is good
gyp verb get node dir target node version installed: 14.8.0
gyp verb build dir attempting to create "build" dir: /Volumes/CodeProjects/parcel-test/node_modules/@parcel/source-map/build
gyp verb build dir "build" dir needed to be created? /Volumes/CodeProjects/parcel-test/node_modules/@parcel/source-map/build
gyp verb build/config.gypi creating config file
gyp verb build/config.gypi writing out config file: /Volumes/CodeProjects/parcel-test/node_modules/@parcel/source-map/build/config.gypi
gyp verb config.gypi checking for gypi file: /Volumes/CodeProjects/parcel-test/node_modules/@parcel/source-map/config.gypi
gyp verb common.gypi checking for gypi file: /Volumes/CodeProjects/parcel-test/node_modules/@parcel/source-map/common.gypi
gyp verb gyp gyp format was not specified; forcing "make"
gyp info spawn /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python
gyp info spawn args [
gyp info spawn args   '/usr/local/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/Volumes/CodeProjects/parcel-test/node_modules/@parcel/source-map/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/usr/local/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/jenswegar/Library/Caches/node-gyp/14.8.0/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/Users/jenswegar/Library/Caches/node-gyp/14.8.0',
gyp info spawn args   '-Dnode_gyp_dir=/usr/local/lib/node_modules/npm/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/Users/jenswegar/Library/Caches/node-gyp/14.8.0/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/Volumes/CodeProjects/parcel-test/node_modules/@parcel/source-map',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
gyp verb command build []
gyp verb build type Release
gyp verb architecture x64
gyp verb node dev dir /Users/jenswegar/Library/Caches/node-gyp/14.8.0
gyp verb `which` succeeded for `make` /usr/bin/make

🔦 Context

Simply trying to get the v2 bundler to work on my machine.

💻 Code Sample

Followed the readme to setup project using npm in an empty folder. Same result with yarn.

🌍 Your Environment

Software	Version(s)
Parcel	beta.1
Node	14.8.0 (via homebrew)
npm/Yarn	6.14.7
Operating System	MacOS Catalina 10.15.6

[DeMoorJasper]

That's pretty annoying, great security from Mac but this would basically block all native modules...

Maybe we can sign these packages in some way, not sure how to do that yet. This seems to be interesting but it seems pretty complicated to setup in ci https://developer.apple.com/developer-id/

Luckily this isn't really a security issue and more of an annoyance over mac os blocking every application by default.

[mischnic]

And you need a 100$/year Apple dev account.

I wonder what other (more popular) native npm packages do