Install fails with yarn

[simoneb]

Trying to build a module which depends on this module on a CI server and always get this:

[15:44:25][npm install] > dtrace-provider@0.8.1 install /srv/TeamCity/buildAgent/work/4b73d52a07e78ef9/node_modules/dtrace-provider
[15:44:25][npm install] > node scripts/install.js
[15:44:25][npm install] 
[15:44:25][npm install] 
[15:44:25][npm install] 
[15:44:25][npm install] Process exited with code 1

It always fails silently and exits with exit code 1. I noticed that sometimes building manually a second time works instead, but not consistently.

Any idea what might be going wrong there?

[batwicket]

I don't know enough to say definitively but yarn added a .lock file to avoid non-deterministic installs. Maybe npm-shrinkwrap helps to reliably reproduce the environment.

[davepacheco]

Without more information, I don't think we can say what's going on here. Does V=1 in the environment include any other information?

[dunnock]

Have similar issue, though it seems it fails to install tap-consumer (Mac OS X 10.12.4):

dtrace-provider max$ yarn install
yarn install v0.24.3
info No lockfile found.
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
error Couldn't find a package.json file in "/Users/max/src/logncode/comae/venus/node_modules/dtrace-provider/node_modules/tap/node_modules/tap-consumer"
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

There is an issue on this where yarn works differently than npm with bundledDependencies yarnpkg/yarn#886

[elyscape]

It looks like the issue is that the install script makes some assumptions about the presence of and location of node-gyp, and those (especially the latter) aren't necessarily true when invoked via yarn. It might be worthwhile to change things like this:

var child = spawn('node-gyp', ['rebuild'], options);
child.on('error', function(err) {
  if (err.code === 'ENOENT') {
    console.error('cannot locate node-gyp');
    process.exit(1);
  } else {
    console.error('unknown error:', err);
    process.exit(1);
  }
});

That is to say, rather than attempting to detect the location of node-gyp yourself, let node figure it out from the PATH. This will allow the script to work with yarn, provided that the user does a global install of node-gyp.

[melloc]

@elyscape It looks like npm prepends the node-gyp-bin directory to the PATH, so relying on searching the PATH should work here. I'll investigate a bit more to make sure changing this doesn't break anything, first, but it looks like it should be fine. That logic was first introduced to fix installation with the npm shipped with homebrew (#52 and #53), but I suspect that this will work with those setups, also.

[elyscape]

@melloc Yeah, as far as I can tell, it should work fine with npm installed by Homebrew. It definitely works with npm shipped with Node.

[elyscape]

@melloc, looks like NPM also exposes npm_config_node_gyp as an environment variable (cf. #70), so we could prefer that and fall back to just executing it directly if that's not present.

Edit: Looks like I was wrong; npm_config_node_gyp is only set when the user explicitly tells NPM to override the path for node-gyp, and the node-gyp binary that NPM provides will properly execute it anyway. As such, we shouldn't need to manually support it.

[melloc]

I've tested these changes with a standalone npm@1.4.9, a standalone npm@5.0.3, and the npm@2.15.1 that ships with node v0.10.48. It looks like node-gyp-bin has been added to the PATH since npm/npm@2605310e, so it's been safe to rely on that here for a while.

I have also tested doing a yarn install in a checked-out bunyan repository pointing at a tarball with my changes with node v7.5.0, and the yarn-0.26.0-20170609.2226.js nightly build, without issue.

Additionally, I've done installs with npm on SmartOS, OS X and FreeBSD 10 with several different versions of node, and run the tests each time, all successfully.

[melloc]

I can't find where exactly it's documented that node-gyp should always be discoverable through the PATH, but I did find:

"install": "node-gyp rebuild":

If there is a binding.gyp file in the root of your package and you haven't defined your own install or preinstall scripts, npm will default the install command to compile using node-gyp.

From here. Elsewhere in the file, there's:

Package scripts run in an environment where many pieces of information are made available regarding the setup of npm and the current state of the process.

Additionally, this page states:

A version of the node-gyp utility is bundled and distributed with Node.js as part of npm. This version is not made directly available for developers to use and is intended only to support the ability to use the npm install command to compile and install Addons.

So it sounds like we should be able to rely on this as being supported by npm. I couldn't find any similar guarantees in the yarn documentation.

[davidjb]

In order to work around the issue, I had to yarn global add node-gyp before yarn install'ing my project so that when it came time for dtrace-provider@0.8.5 to be installed, its build would succeed.

Previous to this, I did not have node-gyp and yarn was silently failing to build the native modules for dtrace-provider, which in turn was causing errors to be outputted like so:

{ Error: Cannot find module './build/default/DTraceProviderBindings'
    at Function.Module._resolveFilename (module.js:469:15)
    at Function.Module._load (module.js:417:25)
    at Module.require (module.js:497:17)
    at require (internal/module.js:20:19)
    at Object.<anonymous> (/Users/alansouza/repo/node_modules/dtrace-provider/dtrace-provider.js:17:23)
    at Module._compile (module.js:570:32)
    at Object.Module._extensions..js (module.js:579:10)
    at Module.load (module.js:487:32)
    at tryModuleLoad (module.js:446:12)
    at Function.Module._load (module.js:438:3) code: 'MODULE_NOT_FOUND' }