Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

npm should have an `npm which binary` or `npm bin binary` command to find executables #3738

Closed
josephg opened this Issue · 19 comments

8 participants

@josephg

If you're writing a makefile or something and need to use installed tools (uglify, coffeescript, etc) which have been installed locally, you often can't find them because of how require paths work.

For example, lets say we have project B which depends on uglify - it uses it as a binary in a make script or a postinstall step or something.

B will try and find uglify either by looking in node_modules/.bin/uglifyjs or using $(npm bin)/uglifyjs.

However, if project A depends on B and uglify, you end up with this directory structure:

A/
A/node_modules/.bin/uglifyjs
A/node_modules/B

Now B can't find uglifyjs. npm bin returns the wrong thing (A/node_modules/B/node_modules/.bin may well exist - it just doesn't have uglifyjs in it).

Solutions:

  • npm could provided a command-line tool to find a binary, for example npm which uglifyjs or npm bin uglifyjs
  • npm could link binaries into node_modules/.bin/X for all projects which depend on the library. In the above example, we could add another symlink in A/node_modules/B/node_modules/.bin/uglifyjs so B can find and run uglifyjs.
@domenic
Collaborator

I believe this is a dupe of #1543.

@domenic domenic closed this
@isaacs
Owner

For what it's worth, if your script is run as an npm package.json "scripts" command, then all your dependencies' bin files are already in the PATH when run. For example:

{
  "name":"coffee-thingie",
  "version":"1.2.3",
  "devDependencies": { "coffee-script": "*" },
  "scripts": { "prepublish": "cake do-stuff" }
}

then doing npm publish or npm install or npm run-script prepublish will call cake do-stuff, and no matter where the bin lives, it'll be in the PATH.

@isaacs
Owner

That is, you should never have to do ./node_modules/.bin/anything, for any reason. If you're running your script from npm, then that's already in the PATH environment variable.

@josephg

Awesome - I didn't know that. Thanks @isaacs!

@josephg

Actually @isaacs - the more I think about this, the more I think forcing scripts to be run through npm scripts is a bad idea. For example, if I want to use a straight-up makefile, I'll need to either invoke my makefile with npm run-script make-all, or more likely write my rules inside npm scripts (npm run-script custom-build-step). Its kind of debilitating.

I think a npm which coffee or npm bin-paths command would be a better interface.

@davidchambers

I just got bitten: davidchambers/xyz#4. I don't want to ship a command-line utility which only works if invoked via npm run-script, so I need another solution. I'm considering doing something like this…

next_version=$(node -p "require('semver').inc('$version', '$increment')")

rather than…

next_version=$("$(npm bin)/semver" -i "$increment" "$version")

solely because require() is guaranteed to work whereas npm bin may print the path to a nonexistent directory. It would be nice to be able to rely on an executable existing (as a symlink, perhaps) at a certain relative path, or to have npm which.

+1

@isaacs
Owner

@davidchambers That will work, but only if you cd "$(dirname "$0")" so that you're actually running the node command relative to the bash script you're describing.

I still don't understand why this is necessary, however.

Why can't you define your bin script as a Node program, and then just use require("semver") to pull it out of your module's dependencies? Why use bash at all?

@isaacs
Owner

Alternatively, xyz can add semver to its bundledDependencies list, and be sure that it's always getting its own copy (of course, this'll break if you dedupe).

@davidchambers

Thanks very much for your comments, @isaacs.

Why can't you define your bin script as a Node program, and then just use require("semver") to pull it out of your module's dependencies? Why use bash at all?

The script makes several calls to git. It's simpler to make these calls from a shell script than to use Node's child_process module. The latter would involve callbacks and manual error checking, whereas the Bash script is synchronous and requires no manual error checking (thanks to set -e).

Alternatively, xyz can add semver to its bundledDependencies list, and be sure that it's always getting its own copy (of course, this'll break if you dedupe).

This sounds like just the ticket! Thanks for bringing it to my attention.

@josephg

I want the feature so I can run make from the commandline and have my makefile find uglify. As it stands, I can:

  • Run make through npm, but npm run-script prepublish is much more awkward than running make
  • npm build ., but thats both awkward and it rebuilds my dependancies (which isn't necessary)
  • I could add an uglify invocation into my package.json then my Makefile could invoke npm run-script uglify. It seems twisty and weird - we'd end up with npm run-script prepublish invokes make invokes npm run-script uglify invokes uglifyjs.

I want to write this:

UGLIFY=`npm which uglifyjs`

foo.min.js: foo.js
  $(UGLIFY) foo.js > foo.min.js

So beautiful. :kissing_heart:

@domenic
Collaborator

What you really want is npm exec, which doesn't exist yet. I would link to the bug, but GitHub's bug autocomplete is failing. #lazydomenic

@josephg

Yeah npm exec would be equally nice. If thats in the works, I'll wait for it. Then it'd be:

UGLIFY="npm exec uglifyjs"

foo.min.js: foo.js
  $(UGLIFY) foo.js > foo.min.js
@davidchambers

It seems twisty and weird

Indeed. That's a good example, @josephg.

@rlidwka

I want the feature so I can run make from the commandline and have my makefile find uglify.

I'm using this instead:

export PATH := ./node_modules/.bin:../node_modules/.bin:../../node_modules/.bin:$(PATH)

It'd be nice to find something better, but npm is way too slow to be executed in a script. Maybe a small separate util?

Maybe bash/zsh can be extended to make recursive lookup in node style?

@timoxley
Collaborator

I've got a module for doing this programatically: https://github.com/timoxley/bin-path

@josephg

@domenic actually, @rlidwka is right - npm has a 300ms startup time. Running npm exec would add 300ms to every node script I run from my makefile, which would easily dominate the calls to uglify. npm which wouldn't have that problem. I would also happily use npm path, which would simply print out the path that npm is adding to my shell scripts.

@rlidwka that depends on the working directory being set to the module directory - which is fine for makefiles, but wouldn't work for other scripts.

@timoxley cool - I might start using that in the meantime.

@davidchambers davidchambers referenced this issue in timoxley/bin-path
Closed

command-line interface #1

@timoxley
Collaborator

The bug @domenic mentioned above is probably: #3313

@timoxley
Collaborator

Also just created a few related modules:

@rlidwka rlidwka referenced this issue from a commit in npm/npm-registry-couchapp
@bcoe bcoe don't use global json. 149662d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.