Child process does not get killed when parent dies by SIGKILL

[surajwy]

• Node.js Version: v8.12.0
• OS: Mac, CentOS
• Scope: runtime
• Module:child_process.js and cluster.js

I create a child process using child_process.fork() method. When I kill parent using kill -9, the child does not die.
Now, If I use the cluster module to create a child process and kill the parent process with -9 signal, all the child dies. Even the cluster modules use child_process.fork() method internally.

So, what is the difference here? Shouldn't OS take care of sending SIGKILL to all children?

CODE

Case 1: Using child_process.fork()
parent.js

const { fork } = require('child_process');

const forked = fork('child.js');

forked.on('message', (msg) => {
        console.log('Message from child', msg);
});

forked.send({ hello: 'world' });

child.js

process.on('message', (msg) => {
        console.log('Message from parent:', msg);
});
let counter = 0;
setInterval(() => {
        try{
              process.send({ counter: counter++ });
        }
        catch(ex){
             // This will come once parent dies. 
             // One way can be to check for error code ERR_IPC_CHANNEL_CLOSED
             //     and call process.exit()
             console.log('parent died', ex.toString());
        }
    }, 1000);

Case 2: Using the cluster module
index.js

let cluster = require("cluster");

if (cluster.isMaster) {
    console.log('cluster settings', cluster.settings);
    cluster.fork();
    console.log('cluster settings', cluster.settings);
    cluster.on('fork', function (worker) {
        console.log('Worker ' + worker.process.pid + ' forked.');
    });
    cluster.on('exit', function (worker) {
        console.log('worker ' + worker.process.pid + ' died. Replacing the died worker...');
        cluster.fork();
    });
}
else {
    setInterval(() => {
        console.log('inside child')
    }, 5000)
}

[ORESoftware]

there are some cases where children dont get the signal..sometimes its an OS thing, try doing this instead (use exec):

$ exec node yours.js

see if the children will die then. otherwise, another simple solution is to have a sigint hander in the parent:

const childPids = [];

process.on('sigint', s => {
  for(let pid of childPids)
    // kill the pid
});

you can get the child pid with:

const k = cp.spawn();
childPids.push(k.pid);

cp.fork() should also have a pid property

[surajwy]

Hi @ORESoftware

Thanks for your suggestions. However, I have a different concern. My question is why the cluster module is doing things right and not the vanilla child_process.fork module.

In the node js cluster module, there is a function which gets executed on the child process which makes some difference when compared with vanilla child_process.fork() method.
Refer: lib/internal/cluster/child.js file and _setupWorker function.

Now, your suggestion is absolutely right to handle general signals. But this will still not handle SIGKILL. SIGKILL is also important to handle because the OOM killer uses SIGKILL which can happen in my case.

To add to your solution, what we can do is, have a ping message from the child process to the master process. If the child gets ERR_IPC_CHANNEL_CLOSED error while doing process.send('ping message');, then we can assume that the parent process is no more. In such a case, we can exit the child process.

Let me know your thought.

[surajwy]

Hi @bnoordhuis,
Any chance you have some time to help me understand this?
Thanks a lot.

[bnoordhuis]

You basically deduced it yourself already. :-)

Cluster workers cannot exist without the master process so they terminate when the master goes away. Processes created with child_process.fork() don't have that restriction.

[surajwy]

Thanks @bnoordhuis, for your swift reply.
If you have some time then can we dig a little deep? There is no urgency on this. You can reply whenever you have time.

My question is:

How the cluster workers get to know that master got killed (by using SIGKILL)?

I have below 3 points in my mind -

1. Master cannot inform it's children that he died. This is because no one can listen to SIGKILL signal. So the master cannot inform children to die.

2. Is OS doing the signaling part? In the case of cluster module, is node.js setting some process level flags like "prctl(PR_SET_PDEATHSIG, SIGKILL);"? Refer: this forum.

3. Is the child worker doing a ping check on the master? Basically is the child on a setInterval doing process.send() call. And if this call fails with ERR_IPC_CHANNEL_CLOSED error, it kills itself.

Thanks @bnoordhuis for your time.

[bnoordhuis]

There are two possible scenarios:

1. You send the SIGKILL to the process group, not just the master process. That will terminate both the master and the workers forcefully. That's like your second option but without the prctl().

2. You send the SIGKILL to the master process. The workers have a bidirectional pipe with the master process that they use for communication. They detect when the master's end is closed (because it got killed) and act accordingly. There's no need to poll at intervals, the kernel notifies the workers when the other end was closed.

[surajwy]

In case of point 2 above, which piece of code in the cluster.js (lib/internal/cluster/master.js or lib/internal/cluster/child.js) registers something with OS Kernel for this pipe close notification.

My whole thought here is that I also need to create master and workers using child_process.fork() and I wish to add this special pipe close notification in worker so as to kill them when the master dies.

1. One way is to blindly go ahead and use cluster (even though in my use case, I don't need a listen() directive).
2. The other way is to pass child_process.fork() options as {"env": {"NODE_UNIQUE_ID": 1}}. By doing this, lib/internal/bootstrap/pre_execution.js function initializeClusterIPC get executed and it does something special and my workes become smart and they die when the master dies. However, this looks hacky to me.

I have tried reading cluster code and was curious how cluster module is doing things right and how do I bring this smartness to my vanilla job worker forked using child_process.fork().

Thanks @bnoordhuis, for your time.

[bnoordhuis]

@surajwy It's the ipc option to child_process.fork().

[gireeshpunathil]

ping @surajwy - is this still outstanding?

[papb]

Have you tried sindresorhus/execa?

[gireeshpunathil]

answered, closing

[Romain-P]

You can do it using native module such as https://github.com/131/node-winapi
I recently pushed a PR for support latest node (12.10.0)

Applying this code on your main.js

var winapi = require('winapi');

winapi.CreateJobGroup();

All child processes created from your app will be automatically killed if your app is killed (even from the task manager)