How to pipe the output forked/cluster processes into the master stdout/stderr

[IonicaBizau]

When doing worker = cluster.fork() I would like to be able to forward the stdout and stderr streams into the master Node.js process.

How can I do that?

I tried doing:

worker = cluster.fork();
worker.process.stdout.pipe(process.stdout)
//             ^ this is null

A workaround would be to use messages, but I want to be able to stream content.

worker.on("message", function(msg){
  console.log("Master says:" + msg);
});
...
worker.send({message:'hello'});

---

How can I access the stdout of the forked process/cluster?

Stackoverflow Question

[ORESoftware]

try this:

worker.stdout.pipe(process.stdout);
worker.stderr.pipe(process.stderr);

[IonicaBizau]

@ORESoftware Sorry, but there are no stdout or stderr properties on worker.

[ORESoftware]

sorry, my bad. the workers are launched in a separate node.js process from master. By default, their stdout/stderr should go to the parent process (I think). Note that process.pid has two different values below.

this is closer, but actually I don't know how to direct the stdout/stderr of the workers to the parent perse:

#!/usr/bin/env node
'use strict';

import * as Cluster from 'cluster';

Cluster.setupMaster({silent:true});

if(Cluster.isMaster){
  
  const w = Cluster.fork();
  console.log('this is the master process:', process.pid);
  w.process.stdout.pipe(process.stdout);
  
  setInterval(() => {}, 1000);
  
}
else{
  console.log('this is the worker process:', process.pid);
}

using {silent:true} allow you to use w.process.stdout.pipe(process.stdout);

[IonicaBizau]

@ORESoftware I'm not sure what I'm missing, but it simply doesn't work:

$ cat t.js 
#!/usr/bin/env node

'use strict';

const Cluster = require('cluster')

Cluster.setupMaster({
    silent: true
});

if (Cluster.isMaster) {
    const w = Cluster.fork();
    console.log('this is the master process:', process.pid);
    w.process.stdout.pipe(process.stdout);
    setInterval(() => {}, 1000);
} else {
    console.log('this is the worker process:', process.pid);
}
$ node t.js 
this is the master process: 67023
^C

[IonicaBizau]

Figured it out: setupMaster should be called in the master process only:

#!/usr/bin/env node

'use strict';

const Cluster = require('cluster')

if (Cluster.isMaster) {
    Cluster.setupMaster({
        silent: true
    });
    const w = Cluster.fork();
    w.process.stdout.pipe(process.stdout);
    console.log('this is the master process:', process.pid);
    setInterval(() => {}, 1000);
} else {
    console.log('this is the worker process:', process.pid);
}

And it works now!

Thanks a lot! If you like, you can post an answer on Stackoverflow too!

[ORESoftware]

for sure np, what are you using the cluster module for by the way? usually cluster module is the first step towards scaling, but when you run servers on AWS, Heroku etc, they will scale and make full use of the hardware.

[IonicaBizau]

That's a good question—yes, for scaling. I was using it on my cloud server, but yesterday I started using pm2—so, I'm not sure if pm2 will do the same thing like cluster does.

[ORESoftware]

yeah this is important. on some systems, cluster might be useful. but on most modern hosting platforms, you will run a server with docker, and basically it will run like 5-10 docker containers on one machine, which means you will use the hardware to the max. So you don't need cluster. You just need one node.js server per docker container.

Like this:

1:1 - node server : docker container
8:1 - docker containers : machine

no need for the cluster module...you can do your own research, but cluster module probably shouldn't have been included in core, I don't know what the purpose is for it.

[IonicaBizau]

Thank you very much!