npm should have an `npm which binary` or `npm bin binary` command to find executables #3738

Closed
josephg opened this Issue Aug 2, 2013 · 21 comments

Comments

Projects
None yet
10 participants

josephg commented Aug 2, 2013

If you're writing a makefile or something and need to use installed tools (uglify, coffeescript, etc) which have been installed locally, you often can't find them because of how require paths work.

For example, lets say we have project B which depends on uglify - it uses it as a binary in a make script or a postinstall step or something.

B will try and find uglify either by looking in node_modules/.bin/uglifyjs or using $(npm bin)/uglifyjs.

However, if project A depends on B and uglify, you end up with this directory structure:

A/
A/node_modules/.bin/uglifyjs
A/node_modules/B

Now B can't find uglifyjs. npm bin returns the wrong thing (A/node_modules/B/node_modules/.bin may well exist - it just doesn't have uglifyjs in it).

Solutions:

  • npm could provided a command-line tool to find a binary, for example npm which uglifyjs or npm bin uglifyjs
  • npm could link binaries into node_modules/.bin/X for all projects which depend on the library. In the above example, we could add another symlink in A/node_modules/B/node_modules/.bin/uglifyjs so B can find and run uglifyjs.

enjalot commented Aug 2, 2013

👍

Member

domenic commented Aug 12, 2013

I believe this is a dupe of #1543.

domenic closed this Aug 12, 2013

Owner

isaacs commented Aug 12, 2013

For what it's worth, if your script is run as an npm package.json "scripts" command, then all your dependencies' bin files are already in the PATH when run. For example:

{
  "name":"coffee-thingie",
  "version":"1.2.3",
  "devDependencies": { "coffee-script": "*" },
  "scripts": { "prepublish": "cake do-stuff" }
}

then doing npm publish or npm install or npm run-script prepublish will call cake do-stuff, and no matter where the bin lives, it'll be in the PATH.

Owner

isaacs commented Aug 12, 2013

That is, you should never have to do ./node_modules/.bin/anything, for any reason. If you're running your script from npm, then that's already in the PATH environment variable.

josephg commented Aug 12, 2013

Awesome - I didn't know that. Thanks @isaacs!

josephg commented Aug 14, 2013

Actually @isaacs - the more I think about this, the more I think forcing scripts to be run through npm scripts is a bad idea. For example, if I want to use a straight-up makefile, I'll need to either invoke my makefile with npm run-script make-all, or more likely write my rules inside npm scripts (npm run-script custom-build-step). Its kind of debilitating.

I think a npm which coffee or npm bin-paths command would be a better interface.

I just got bitten: davidchambers/xyz#4. I don't want to ship a command-line utility which only works if invoked via npm run-script, so I need another solution. I'm considering doing something like this…

next_version=$(node -p "require('semver').inc('$version', '$increment')")

rather than…

next_version=$("$(npm bin)/semver" -i "$increment" "$version")

solely because require() is guaranteed to work whereas npm bin may print the path to a nonexistent directory. It would be nice to be able to rely on an executable existing (as a symlink, perhaps) at a certain relative path, or to have npm which.

+1

Owner

isaacs commented Apr 16, 2014

@davidchambers That will work, but only if you cd "$(dirname "$0")" so that you're actually running the node command relative to the bash script you're describing.

I still don't understand why this is necessary, however.

Why can't you define your bin script as a Node program, and then just use require("semver") to pull it out of your module's dependencies? Why use bash at all?

Owner

isaacs commented Apr 16, 2014

Alternatively, xyz can add semver to its bundledDependencies list, and be sure that it's always getting its own copy (of course, this'll break if you dedupe).

Thanks very much for your comments, @isaacs.

Why can't you define your bin script as a Node program, and then just use require("semver") to pull it out of your module's dependencies? Why use bash at all?

The script makes several calls to git. It's simpler to make these calls from a shell script than to use Node's child_process module. The latter would involve callbacks and manual error checking, whereas the Bash script is synchronous and requires no manual error checking (thanks to set -e).

Alternatively, xyz can add semver to its bundledDependencies list, and be sure that it's always getting its own copy (of course, this'll break if you dedupe).

This sounds like just the ticket! Thanks for bringing it to my attention.

josephg commented Apr 16, 2014

I want the feature so I can run make from the commandline and have my makefile find uglify. As it stands, I can:

  • Run make through npm, but npm run-script prepublish is much more awkward than running make
  • npm build ., but thats both awkward and it rebuilds my dependancies (which isn't necessary)
  • I could add an uglify invocation into my package.json then my Makefile could invoke npm run-script uglify. It seems twisty and weird - we'd end up with npm run-script prepublish invokes make invokes npm run-script uglify invokes uglifyjs.

I want to write this:

UGLIFY=`npm which uglifyjs`

foo.min.js: foo.js
  $(UGLIFY) foo.js > foo.min.js

So beautiful. 😘

Member

domenic commented Apr 16, 2014

What you really want is npm exec, which doesn't exist yet. I would link to the bug, but GitHub's bug autocomplete is failing. #lazydomenic

josephg commented Apr 16, 2014

Yeah npm exec would be equally nice. If thats in the works, I'll wait for it. Then it'd be:

UGLIFY="npm exec uglifyjs"

foo.min.js: foo.js
  $(UGLIFY) foo.js > foo.min.js

It seems twisty and weird

Indeed. That's a good example, @josephg.

Contributor

rlidwka commented Apr 17, 2014

I want the feature so I can run make from the commandline and have my makefile find uglify.

I'm using this instead:

export PATH := ./node_modules/.bin:../node_modules/.bin:../../node_modules/.bin:$(PATH)

It'd be nice to find something better, but npm is way too slow to be executed in a script. Maybe a small separate util?

Maybe bash/zsh can be extended to make recursive lookup in node style?

Member

timoxley commented Apr 17, 2014

I've got a module for doing this programatically: https://github.com/timoxley/bin-path

josephg commented Apr 17, 2014

@domenic actually, @rlidwka is right - npm has a 300ms startup time. Running npm exec would add 300ms to every node script I run from my makefile, which would easily dominate the calls to uglify. npm which wouldn't have that problem. I would also happily use npm path, which would simply print out the path that npm is adding to my shell scripts.

@rlidwka that depends on the working directory being set to the module directory - which is fine for makefiles, but wouldn't work for other scripts.

@timoxley cool - I might start using that in the meantime.

davidchambers referenced this issue in timoxley/bin-path Apr 17, 2014

Closed

command-line interface #1

Member

timoxley commented Apr 21, 2014

The bug @domenic mentioned above is probably: #3313

Member

timoxley commented Apr 21, 2014

Also just created a few related modules:

@rlidwka rlidwka referenced this issue in npm/npm-registry-couchapp Sep 26, 2014

@bcoe @isaacs bcoe + isaacs don't use global json. 149662d

kuraga commented May 16, 2015

Updates?

yinrong commented Dec 19, 2015

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment