Passing flags and arguments into run-script

[ELLIOTTCABLE]

At the moment, there's really no way to pass arguments to scripts specified in the "scripts" field with npm run-script.

I have three options, currently:

1. Hard-code the flags into the "scripts" field. This obviously doesn't cover all situations; there's plenty of situations where I want to vary options I pass to some of the scripts, or make the scripts I write more widely applicable / configurable.
2. Pack the flags into an environment variable. This isn't very composable, it's quite un-UNIX-y, and it requires me to define all of the flags in one place. (Not to mention that WITH="--debug --optimize" npm run-scripts compile just ain't very pretty. ;)
3. Wrap the script in question (if it's not my own), and query the user interactively. Again, not remotely composable or UNIX-y; and to boot, means I have to interact with the script every single time I run it.

---

Here's my suggestion:

• -fLGs and --flags that come after the second extant argument to run-script are appended to the string in the "scripts" field before it's run
• anything after -- is appended to the string in the "scripts" field, before it's run

Examples:

• npm run-scripts test -- something/something_else.js
• npm run-scripts a_module compile --debug --optimize
• npm run-scripts clean -- -dXf

This is completely backwards-compatible with what exists now; it does mean that we'd have to explicitly name the current module if we didn't use --, but that's minor, when it guarantees backwards compatibility.

[mfncooper]

There's a good deal of flexibility already. Take a look at:

• The config section in npm help json
• The ENVIRONMENT section in npm help scripts
• The Per-Package Config Settings section in npm help config

[domenic]

I agree this would be nice; it's a pain to switch from npm test to npm install mocha -g; mocha --grep "stuff" or node node_modules/.bin/mocha --grep "stuff" just to use a feature of my test framework. I'd love @isaacs to chime in on this idea.

[ELLIOTTCABLE]

@mfncooper I'm using all of those, yes; but the current design is extremely un-UNIXy. Neither do any of those help with the particular use-case @domenic and I are talking about. (=

[luk-]

-1

[ELLIOTTCABLE]

@st-luke reasoning? Commentary?

[luk-]

I think it's additional complexity that's totally unnecessary.

On Tuesday, June 4, 2013, ELLIOTTCABLE wrote:

@st-luke https://github.com/st-luke reasoning? Commentary?

—
Reply to this email directly or view it on GitHubhttps://github.com/isaacs/npm/issues/3494#issuecomment-18904446
.

[ELLIOTTCABLE]

@st-luke … and given the use-case put forward by myself, and echoed by @domenic, could you please explain how it's still “totally unnecessary?”

[luk-]

@ELLIOTTCABLE if it's special npm functionality/complexity to facilitate using the feature a module that is meant to be installed globally, then I think it seems incredibly silly to add this in. We should wait for the final word from @isaacs, but I truly believe there are other and better ways to do what you are trying to do than to shove this kind of code into the package manager.

[domenic]

to facilitate using the feature a module that is meant to be installed globally

That's the thing though: installing testing runners globally is a nightmare. Much better to use the version bundled with the package you're developing/testing, which is precisely what npm test does.

[luk-]

That's the thing though: installing testing runners globally is a nightmare.

Agreed, there are multiple good reasons to not do this.

Much better to use the version bundled with the package you're developing/testing, which is precisely what npm test does.

Agreed, why not require('mocha') in your test script and use the features you need from that?

Maybe I'm missing something, but what's the advantage of adding this feature other than saving you a small amount of code in your test files or scripts themselves?

[domenic]

@st-luke Here's the scenario. I have npm test set up to run mocha, which because of "scripts" works great. It runs the local copy in node_modules/.bin/mocha. This is great!

Now I am happily developing, and I run my tests with npm test. But one day I want to narrow it down to only the tests I am working on, e.g. the tests with "foo" in their name. If I were using the global mocha command, I would do

$ mocha --grep "foo"

But I am being a good boy and am using the locally installed mocha, via npm test. So I try

$ npm test --grep "foo"

but as discussed in this bug, this doesn't work! So what are my workarounds? Either (a) I have to either revert to using the global mocha install, with all its perils, or (b) I have to look at the contents of my "scripts" hash, find mocha there, then mentally translate that to node node_modules/.bin/mocha, then finally type

$ node node_modules/.bin/mocha --grep "foo"

which is pretty horrid and abstraction-boundary-breaking. In both cases I lose the benefits of the "scripts" hash.

The proposal in the OP is to just allow

$ npm test -- --grep "foo"

which seems reasonable to me.

[luk-]

but as discussed in this bug

(this isn't a bug)

So this is mainly for mocha? How many other situations is this applicable to?

Because these examples from the original post:

npm run-scripts a_module compile --debug --optimize
npm run-scripts clean -- -dXf

are inappropriate use of npm's scripts.

[ELLIOTTCABLE]

@st-luke that's my bad, then, for not taking long enough to come up with better examples for the original report I posted. The first example, I can see how you'd consider it inappropriate. I was only using them as examples of my proposed syntax; I made the mistake of assuming that nobody sane would dispute the usefulness of the concept itself.

As for examples of the use, then, let's focus on @domenic's example. It's quintessential and applicable.

[domenic]

So this is mainly for mocha? How many other situations is this applicable to?

It's applicable to pretty much every test runner I've used.

More generally, given the niceness of npm's scripts as a cross-platform alternative to make etc., it can serve well for many situations. Two that I've run into personally:

• node node_modules/.bin/grunt build:production vs. the proposed npm run build -- --production
• node node_modules/.bin/jshint --verbose vs. the proposed npm run lint -- --verbose

[ELLIOTTCABLE]

Basically, it boils down to this: there's currently no UNIX-y way to argument scripts in npm. You have to modify the script (whether modifying the invocation in package.json, or duplicating the script itself so you've got two versions that operate slightly differently) to modify the behavior slightly. There's more than one way to npm test your codebase (with coverage instrumentation, or not? compile the client-side tests and open them in the system's browser, too? start a file-watcher so the tests re-run each time you change one of the source-files in realtime? generate pipe-able output, or user-friendly coloured output?), so there should be more than one way to invoke npm test.

The only alternatives are either multiple "scripts" fields for the same actual purpose (and you can't really be suggesting that one have npm test-with-coverage, npm test-and-compile-client-tests, npm test-with-coverage-and-compile-client-tests, npm test-with-coverage-with-colour, npm test-with-coverage-and-colour-and-watch …), or aping flags via environment variables or temporarily changing configuration variables, both of which are absurdly un-UNIX-y / messy.

[ELLIOTTCABLE]

Not that it's especially relevant in a pure, abstract way (as npm has a different purpose than any of these) … but since it does demonstrate that something like this is desirable, all of the following have a method for this.

Make

target:
        $(MAKE) --something=$(FOO) --something-else=$(BAR)

---

$ make target FOO=123 BAR=456

Rake

task :target, :foo, :bar do |args|
   do_something args.foo + args.bar
end

---

$ rake target[123,456]

Cake

option '-f', '--foo [VAL]'
option '-b', '--bar [VAL]'

task 'target', 'Blah blah!', (options) ->
   console.log options.foo + options.bar

---

$ cake target --foo 123 --bar 456

Grunt

grunt.registerTask('target', "Blah blah blah.", function(foo, bar) {
  console.log(foo + bar);
});

---

$ grunt target:123:456

Apache Ant

<target name="target">
   <java executable="something.exe">
      <arg value="${foo}"/>
      <arg value="${bar}"/>
   </java>
</target>

---

$ ant -Dfoo=123 -Dbar=456 target

[luk-]

Basically, it boils down to this: there's currently no UNIX-y way to argument scripts in npm. You have to modify the script (whether modifying the invocation in package.json, or duplicating the script itself so you've got two versions that operate slightly differently) to modify the behavior slightly. There's more than one way to npm test your codebase (with coverage instrumentation, or not? compile the client-side tests and open them in the system's browser, too? start a file-watcher so the tests re-run each time you change one of the source-files in realtime? generate pipe-able output, or user-friendly coloured output?), so there should be more than one way to invoke npm test.

It sounds like you just want a makefile.

[ELLIOTTCABLE]

@st-luke that doesn't solve the problem that the go-to invocation for all module users, is going to be npm test or similar; in fact, the npm documentation says that this should be the case. (Not to mention that I don't want to make autotools/make a development dependency of my JavaScript project.)

npm's approach is great: just make shell scripts, and use the package.json as a central place to list/name them, allowing npm as a central target for invoking them. Unfortunately, in the particular respect documented by this bug-report, this is inflexible / untenable.

[evantahler]

I would offer an alternative reason to want this feature:

Say you have a framework like Sails or ActionHero, and you want provide tools for folks to generate files and run a server. If folks install the package globally (npm install -g), the contents of the project's ./bin become available in the $PATH and are generally available. However, I'm under the impression that installing packages globally should be avoided, especially when different projects might require different versions.

I know I can run "local" binaries with ./node_modules/.bin/{myBinary}, but that is a rather tedious collection of commands for newcomers.

Global

• npm install -g myPackage
• myPackage generate --name=thing
• myPackage start server

Local

• npm install myPackage
• ./node_modules/.bin/myPackage generate --name=thing
• ./node_modules/.bin/myPackage start server

What is being discussed here would be:

• npm install myPackage
• npm run-script myPackage generate --name=thing
• npm run-script myPackage start server

The run-script (or run) option is a nice solution which

1. doesn't require a global install which is likely to confuse folks down the line when they need to change versions
2. doesn't require the creation of a "binary"
3. doesn't require knowledge of "hidden" NPM directories

---

Ideas copied in from the discussion here https://groups.google.com/forum/?fromgroups#!msg/npm-/XR5sRLt_9gk/hulhvU44SKQJ

[ELLIOTTCABLE]

@evantahler not sure I'm understanding how what you're talking about, is an alternative to what I'm talking about. Are you proposing adopting what I'm suggesting (from your --name=thing example, I feel that this is so) and further suggesting that we dispose of the "scripts" field in favour of ./bin files? Or what?

[evantahler]

My example was probably more convoluted than I intended, sorry. I'm agreeing with @ELLIOTTCABLE in asking for the ability to pass argument to run-scrpt, and demonstrating an alternative use case to @domenic, one that isn't about testing (which might be simper to discus as there are less dependancies).

Basically, I would love to be able to pass arguments into run-script to use generators. As a module developer, I don't really have a preference whether or not I create "scripts" or "binaries" (as they are just node *.js files in my case).

As a user of the module, it seems that there are only 2 ways to execute code from a module which can take arguments:

1. global install with globally available binaries (myPackage generate --name=thing)
2. local install and access at ./node_modules/.bin/* (./node_modules/.bin/myPackage generate --name=thing)

I think both of these options would be less user-friendly being able to pass arguments to run-script for the reasons above. I posit that npm run-script myPackage generate --name=thing would be a great alternative to # 1 and # 2

[domenic]

FYI @isaacs says this should probably be implemented after #3313 since that framework will definitely be used.

[ELLIOTTCABLE]

@domenic @isaacs I ran into exactly #3313 when trying to implement a proof-of-concept of this, ripping out the relevant bits of bin/npm, run-script.js, and lifecycle.js. Can't wait for that to be completed.

I'm all for waiting on #3313, then this issue's suggested changes would be an absolute breeze. (=

[hisaboh]

+1

[pdonald]

+1

I'm also using mocha and would like to pass --grep test. I'm using npm test instead of mocha because with coffee script arguments the mocha command becomes very long. When I'm on Windows I don't know how to use aliases in cmd.exe so now I manually edit package.json every time I need to add --grep.

[statico]

+1

Supporting this would make our, and presumably a lot of other peoples', lives easier. Arguments against this seem to be unnecessary mandates.

[jspiro]

+1

[medikoo]

Definitely +1.

I don't use Makefile as I want to use something that's environment agnostic (not just for *nix systems). I don't use grunt as I already use npm which provides very close functionality with scripts. The only thing I miss from npm scripts is ability to pass the optional arguments to underlying scripts.

The best way to achieve it, would probably be to append anything after -- to the command mentioned in scripts field (assuring it will also work on Windows).

[fidian]

+1

Currently I use npm test which runs a shell script that will execute each part of the testing suite we have. To run them separately, there's scripts: npm run-script vows, npm run-script karma, npm run-script e2e, etc. Yes, we have multiple testing frameworks and so invoking each one with the right flags gets tedious. I'd love to be able to combine the tests into a single shell script so I can use npm test or npm test --vows or npm test --karma, and so forth. Adding this, even if this requires special syntax, would be tremendously helpful. For instance, you might want to say "arguments go here" in the package.json file like this, borrowing a bit from bash: scripts: { "test": "util/bin/run_tests $*" }

[ghost]

What is the current status on this?

[rlidwka]

@bucaran , it's done. Sorta.

You can run npm test -- -t 1234, and it will append -t 1234 to the end of your script.

You still can't put an argument in the middle of the script though.

[ghost]

@rlidwka That does it! Thanks.

[Morriz]

Any updates on this? I still can't do
npm run blah -- --someparam
;(

[whymarrh]

@Morriz what version of npm are you using? Try running npm update -g npm.

[Morriz]

I am on version 1.4.28 because nvm installed it with node 0.10.38...tnx for pointing it out ;P

On 21 apr. 2015, at 12:11, Whymarrh Whitby notifications@github.com wrote:

@Morriz https://github.com/Morriz what version of npm are you using? Try running npm update -g npm

—
Reply to this email directly or view it on GitHub #3494 (comment).

[ELLIOTTCABLE]

For those of you, like @Morriz, waiting on this feature and using nvm, you may have run into the same issue I did when trying to use this partial-implementation: npm won't update itself under nvm if there's a conflicting .npmrc: nvm-sh/nvm#606

[IngwiePhoenix]

This is a very long issue and I read through half of it, reading the original proposal a nd am already agreeing to this. I am at the point where I want ot make locally bundled tools available through scripts so I can use them in internal hooks. My app has a self-updating mechanism it can use during development. Another similar service runs on my server that is triggered during deploy. It would be convenient to use my copy of bower as: npm bower -- [bower args].

What is the current official standing on this and what are current workarounds/solutions? I am going to look into nvm now.

[whymarrh]

@IngwiePhoenix I'm the farthest thing from an official voice, but this is available in the current version of npm. You should be able to run npm run bower -- $args and have that work fine. Your package.json file would contain a bower entry in the scriptsobject. See also run-script.

[ghost]

@whymarrh Yes, but the problem is you can't reuse the parameters inside / the middle of your script via variables $1, $2, ...

[IngwiePhoenix]

@whymarrh @bucaran Thanks! That actually is what I was looking for. $1 and $N are not much of an interest for me. And actually, why would it? You could write a small bit of code, pass your arguments and then proxy them from within your script. After all, it gets them in process.argv. So you can still re-arrange them and run if you would like to. It may be a bit hacky but could work. :)

[ghost]

@IngwiePhoenix Good point. It would be useful if this was more transparent.

[IngwiePhoenix]

@bucaran Yeah, I hear you there. I am currently setting up to use my local bower and webpack installs for some hooks in my deployment system - way easier. :) But for now the way its implemented, its totally usable and pretty nice by itself.

To show a little of the method I mentioned earlier:

var argv = process.argv;
var proc = require("child_process").spawn("myCommand", [
    "--target="+argv[1],
    // ...
]);

Even if hacky, it'd be one way to work around this. I am using the yargs module to handle args in my case. :)

Thanks to the NPM devs for working on this!

[zladuric]

+1 for the reusable arguments here, if it's going to happen.

[kyriacos]

+1 for reusable arguments :)

[ELLIOTTCABLE]

So, despite originally championing this, I'm 👎 on these requests for argument-reordering. This is, from my point of view, unnecessary complication; and will lead to often-changing, and even buggy(!!) package.json files … I think this is all a terrible idea. Add an actual script to your project, a versioned support file, and point to that, if you need to do more complicated invocation of scripts.

Just my 2¢.

Postscript: If you do want this, maybe it's time to open a new Issue? I think we can close this issue as completed.

[othiym23]

@ELLIOTTCABLE This issue has been closed since last August. ;) I agree with the sentiment, though – further bikeshedding feature requests in the comments on a closed issue is probably not going to get those features any closer to being implemented.

[ericsoco]

FWIW... npm run myscript -- --foo bar is exactly what I was looking for. Google brought me here and I am happy.

[jcollum]

I was looking for a way to run -g with a Lab (Hapi.js) test and the -- -g pattern option worked for me.

package.json:

"test": "gulp transpile; ./node_modules/.bin/lab -r console -e development -l ./dist/test/unit.js",

bash:

npm run test -- -g increm

I think this would also work with Mocha. But any command line tool that mandates that the file argument be at the end of the line wouldn't work here. Can't think of one like that off the top of my head.

Damn it would be nice if I could put a comment in my package.yaml file so that other people using the scripts would know how to use that script to run just one test. #3336