passing unix signals to child rather than dying?

[evantahler]

Here's some interesting behavior I've seen when running an application which does some slower shutdown operations when receiving a termination signal.

When running npm start in a *nix terminal (tested on ubuntu and OSX), the SIGINT (ctrl+c) command is passed from the parent npm process to the child (my running application). npm instantly exits, and then my application handles the passed-down signal and eventually exits. However, because the stdout/err of the session was managed by npm, strange things can happen if my application still writes (or console.outs).

At the very least, the terminal which started the application will interpret the npm process' death as a return and draw the next line. However, the child's stdout is still bound to that session. This means you can start another application in that same terminal and still get the first app's messages intermingled. Or, more strangely, if you then close that terminal session, the underlying app which is still running and possibly trying to write to stdout is likely to crash on the next console.log.

You can emulate this pretty simply:

package.json

{
  "name": "test",
  "scripts": {
    "start": "node ./app.js"
  }
}

app.js

process.on('SIGINT',  function(){ console.log("SIGINT");  shutDown() });
process.on('SIGTERM', function(){ console.log("SIGTERM"); shutDown() });

var string = ".";

var shutDown = function(){
  console.log("off-ing...");
  string = "x";

  setTimeout(function(){
    console.log("bye!");
    process.exit();
  }, 1000 * 60);
}

setInterval(function(){
  console.log(string);
}, 500)

1. start app.js with npm start
2. ctrl+c in the window you started npm with. You will see npm exit, but still receive messages from app.js.
3. have fun opening vi or some other application in the same window.

I think a possible solution to this would to have npm catch (process.on('x')) and not actually exit until the child process exits and pass those commands along to the child... but I'm sure that will cause all sorts of other problems.

Thoughts?

[markhuge]

+1

This becomes an issue when you use npm start in your system init scripts.

It's pretty common to handle logging with npm start >> /var/log/myapp.log 2>&1. This breaks on SIGINT with npm, because the stdout was attached to npm and not the node process.

This also breaks things like upstart, which watch the PID of the commands they invoke. It'll see the process as down, when really it's just npm that's exited.

[othiym23]

It's arguable that this isn't something that npm itself should try to handle – or at least I'm not sure that I, as an ops engineer, would want npm trying to act as a proxy for OS signals to the child application. Upstart and systemd can be configured to watch not just npm but its child processes, although that isn't the way either behaves by default. In general, I'd want the process monitoring system running the node services directly (to minimize overhead, if nothing else).

I don't think the intent with npm start was that it would keep the process running as a child process of npm. Instead, it would boot the process, which would daemonize itself and then run in the background, returning control to the shell. I'm not saying that the current situation is ideal, just that the assumptions behind the implementation of the feature may be out of whack with user expectations. The question then is how to reconcile this conflict, and my inclination is to fix it with documentation tweaks: I don't think it makes sense to make npm a fully-fledged process monitor when things like pm2 and forever exist. Is there a way to get what you want without increasing npm's scope?

[evantahler]

Yeah... I agree with @othiym23's thoughts for production apps.

This is more of a development issue for me with actionhero. It's super weird to signal your server to shutdown, get the new-line, and then a few seconds later, see more output.

I use monit to run my node apps in production, and never use NPM to start them. I always call the executables directly, ie cd /www/app && ./node_modules/.bin/actionhero startCluster --daemon --workers=4. This way, I can have fine-grain control over the pidfile, etc to avoid problems like @markhuge mentions. I actually don't see this as a failure of NPM. It just seems redundant to call NPM if the application can boot itself.

How common is the NODE_ENV environment variable? Common enough for NPM to change behavior based on it?

[markhuge]

@othiym23 That makes sense and I agree with you 100%. This is a best practices/docs issue. 👍 For upstart in particular it makes sense to just expect fork and not make this npm's concern.

In practice, as long as the thing that catches your signal eventually exits, everything still works as expected with npm start (which is probably why I never noticed it before).

I think the disconnect is in what the intended use case of npm start is in the docs. Right now the docs on npm start are:

This runs a package's "start" script, if one was provided.

Is there an appropriate place to submit a PR to update the docs? I saw npm/docs.npmjs.com but that seems to be something else.

[othiym23]

Is there an appropriate place to submit a PR to update the docs? I saw npm/docs.npmjs.com but that seems to be something else.

The docs are up in the air as we try to sort out what goes where. For now, the best place to land doc PRs for stuff that's currently documented in npm's command-line help, manpages, or the webpages on npmjs.org/doc/ is a PR on this very repo. Some of the more concept-orient docs will probably migrate to docs.npmjs.com over time. In general, don't let analysis paralysis get you down – submit patches, and we as a team can figure out where they go. ;)

[vvo]

I also find it disturbing that sending SIGTERM to an npm start(ed) program does not kill the program as it would if it was started with node index.js.

It makes using things like watch_program jsfiles -- npm start impossible. You have to watch_program jsfiles -- node startscript.js.

In this situation, you better not use npm start at all because you will have to maintain two start script commands.

[stuartpb]

Okay, this is a fairly significant problem for me, as it means that any apps I'm developing to use npm start don't get stopped when they're supposed to (in other words, every deploy I do keeps me waiting for 10 more seconds than it should as it waits for the app to respond to a SIGTERM it never gets, then force-kills the entire process group).

Does Node not provide any mechanism for accessing the process-replacing exec() call used by bash -c? (In other words, https://github.com/jprichardson/node-kexec)