3.2.x cluster processes unable to bind to tcp listener port #3943

HughePaul · 2018-10-05T10:01:52Z

What's going wrong?

When starting pm2 in cluster mode on a multi-core machine, only one process is able to bind to a TCP listening port (express listen), the other processes report an Error: listen EADDRINUSE
Started using:

pm2 start app.js --name myname --no-daemon -i max

This appears to start 4 processes in fork mode with pm2 3.2.x, but it starts for processes in cluster mode in pm2 3.1.3

How could we reproduce this issue?

mkdir pm2test
cd pm2test
npm init
npm install express

npm install pm2@3.2

app.js:

const express = require('express');
let app = express();
app.listen(8080);

./node_modules/.bin/pm2 start app.js --name myname --no-daemon -i max
./node_modules/.bin/pm2 kill

npm install pm2@3.1.3

./node_modules/.bin/pm2 start app.js --name myname --no-daemon -i max
./node_modules/.bin/pm2 kill

Supporting information

===============================================================================
--- PM2 REPORT (Fri Oct 05 2018 10:55:38 GMT+0100 (BST)) ----------------------
===============================================================================
--- Daemon -------------------------------------------------
pm2d version         : 3.2.1
node version         : 8.9.4
node path            : ./node_modules/.bin/pm2
argv                 : /home/pwinkler/.nvm/versions/node/v8.9.4/bin/node,/home/pwinkler/projects/test/pm2test/node_modules/.bin/pm2,start,app.js,--name,myname,--no-daemon,-i,max
argv0                : node
user                 : pwinkler
uid                  : 1001
gid                  : 1001
uptime               : 0min
===============================================================================
--- CLI ----------------------------------------------------
local pm2            : 3.2.1
node version         : 8.9.4
node path            : ./node_modules/.bin/pm2
argv                 : /home/pwinkler/.nvm/versions/node/v8.9.4/bin/node,/home/pwinkler/projects/test/pm2test/node_modules/.bin/pm2,report
argv0                : node
user                 : pwinkler
uid                  : 1001
gid                  : 1001
===============================================================================
--- System info --------------------------------------------
arch                 : x64
platform             : linux
type                 : Linux
cpus                 : Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz
cpus nb              : 4
freemem              : 1005617152
totalmem             : 16704774144
home                 : /home/pwinkler
===============================================================================
--- PM2 list -----------------------------------------------
┌──────────┬────┬─────────┬──────┬───────┬────────┬─────────┬────────┬───────┬───────────┬──────────┬──────────┐
│ App name │ id │ version │ mode │ pid   │ status │ restart │ uptime │ cpu   │ mem       │ user     │ watching │
├──────────┼────┼─────────┼──────┼───────┼────────┼─────────┼────────┼───────┼───────────┼──────────┼──────────┤
│ myname   │ 0  │ 1.0.0   │ fork │ 28038 │ online │ 0       │ 0s     │ 28.8% │ 40.3 MB   │ pwinkler │ disabled │
│ myname   │ 1  │ 1.0.0   │ fork │ 28091 │ online │ 2       │ 0s     │ 0%    │ 33.1 MB   │ pwinkler │ disabled │
│ myname   │ 2  │ 1.0.0   │ fork │ 28097 │ online │ 2       │ 0s     │ 0%    │ 33.1 MB   │ pwinkler │ disabled │
│ myname   │ 3  │ 1.0.0   │ fork │ 28103 │ online │ 2       │ 0s     │ 0%    │ 31.5 MB   │ pwinkler │ disabled │
└──────────┴────┴─────────┴──────┴───────┴────────┴─────────┴────────┴───────┴───────────┴──────────┴──────────┘
===============================================================================
--- Daemon logs --------------------------------------------
┌──────────┬────┬─────────┬──────┬───────┬────────┬─────────┬────────┬─────┬───────────┬──────────┬──────────┐
│ App name │ id │ version │ mode │ pid   │ status │ restart │ uptime │ cpu │ mem       │ user     │ watching │
├──────────┼────┼─────────┼──────┼───────┼────────┼─────────┼────────┼─────┼───────────┼──────────┼──────────┤
│ myname   │ 0  │ 1.0.0   │ fork │ 28038 │ online │ 0       │ 0s     │ 0%  │ 40.3 MB   │ pwinkler │ disabled │
│ myname   │ 1  │ 1.0.0   │ fork │ 28091 │ online │ 2       │ 0s     │ 0%  │ 33.1 MB   │ pwinkler │ disabled │
│ myname   │ 2  │ 1.0.0   │ fork │ 28097 │ online │ 2       │ 0s     │ 0%  │ 33.1 MB   │ pwinkler │ disabled │
│ myname   │ 3  │ 1.0.0   │ fork │ 28103 │ online │ 2       │ 0s     │ 0%  │ 31.5 MB   │ pwinkler │ disabled │
└──────────┴────┴─────────┴──────┴───────┴────────┴─────────┴────────┴─────┴───────────┴──────────┴──────────┘
 Use `pm2 show <id|name>` to get more details about an app

===============================================================================
--- PM2 REPORT (Fri Oct 05 2018 10:58:21 GMT+0100 (BST)) ----------------------
===============================================================================
--- Daemon -------------------------------------------------
pm2d version         : 3.1.3
node version         : 8.9.4
node path            : ./node_modules/.bin/pm2
argv                 : /home/pwinkler/.nvm/versions/node/v8.9.4/bin/node,/home/pwinkler/projects/test/pm2test/node_modules/.bin/pm2,start,app.js,--name,myname,--no-daemon,-i,max
argv0                : node
user                 : pwinkler
uid                  : 1001
gid                  : 1001
uptime               : 0min
===============================================================================
--- CLI ----------------------------------------------------
local pm2            : 3.1.3
node version         : 8.9.4
node path            : ./node_modules/.bin/pm2
argv                 : /home/pwinkler/.nvm/versions/node/v8.9.4/bin/node,/home/pwinkler/projects/test/pm2test/node_modules/.bin/pm2,report
argv0                : node
user                 : pwinkler
uid                  : 1001
gid                  : 1001
===============================================================================
--- System info --------------------------------------------
arch                 : x64
platform             : linux
type                 : Linux
cpus                 : Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz
cpus nb              : 4
freemem              : 869748736
totalmem             : 16704774144
home                 : /home/pwinkler
===============================================================================
--- PM2 list -----------------------------------------------
┌──────────┬────┬─────────┬───────┬────────┬─────────┬────────┬──────┬───────────┬──────────┬──────────┐
│ App name │ id │ mode    │ pid   │ status │ restart │ uptime │ cpu  │ mem       │ user     │ watching │
├──────────┼────┼─────────┼───────┼────────┼─────────┼────────┼──────┼───────────┼──────────┼──────────┤
│ myname   │ 0  │ cluster │ 28797 │ online │ 0       │ 2s     │ 0.6% │ 41.8 MB   │ pwinkler │ disabled │
│ myname   │ 1  │ cluster │ 28803 │ online │ 0       │ 2s     │ 0.6% │ 41.7 MB   │ pwinkler │ disabled │
│ myname   │ 2  │ cluster │ 28813 │ online │ 0       │ 2s     │ 5.5% │ 41.8 MB   │ pwinkler │ disabled │
│ myname   │ 3  │ cluster │ 28823 │ online │ 0       │ 1s     │ 8.3% │ 41.5 MB   │ pwinkler │ disabled │
└──────────┴────┴─────────┴───────┴────────┴─────────┴────────┴──────┴───────────┴──────────┴──────────┘
===============================================================================
--- Daemon logs --------------------------------------------
┌──────────┬────┬─────────┬───────┬────────┬─────────┬────────┬─────┬───────────┬──────────┬──────────┐
│ App name │ id │ mode    │ pid   │ status │ restart │ uptime │ cpu │ mem       │ user     │ watching │
├──────────┼────┼─────────┼───────┼────────┼─────────┼────────┼─────┼───────────┼──────────┼──────────┤
│ myname   │ 0  │ cluster │ 28797 │ online │ 0       │ 2s     │ 0%  │ 41.8 MB   │ pwinkler │ disabled │
│ myname   │ 1  │ cluster │ 28803 │ online │ 0       │ 2s     │ 0%  │ 41.7 MB   │ pwinkler │ disabled │
│ myname   │ 2  │ cluster │ 28813 │ online │ 0       │ 2s     │ 0%  │ 41.8 MB   │ pwinkler │ disabled │
│ myname   │ 3  │ cluster │ 28823 │ online │ 0       │ 1s     │ 0%  │ 41.5 MB   │ pwinkler │ disabled │
└──────────┴────┴─────────┴───────┴────────┴─────────┴────────┴─────┴───────────┴──────────┴──────────┘
 Use `pm2 show <id|name>` to get more details about an app

The text was updated successfully, but these errors were encountered:

ufukd · 2018-10-05T10:15:25Z

same issue :(

vinogradov-m · 2018-10-05T14:41:50Z

I faced the similar issue when updated from v3.1.3 to v3.2.0: the app starts fine but it doesn't release a port on reload.
The reason is, somewhere app.exec_mode is compared with string 'cluster_mode', not just 'cluster'.
Earlier there was a function checkExecMode() which sets exec_mode to 'cluster_mode' if it was just 'cluster'. But it was removed.

So I defined exec_mod in my config file as 'cluster_mode' and everything works fine. You can try the same workaround.

Unitech · 2018-10-05T15:09:12Z

wip

Unitech · 2018-10-05T15:10:44Z

I've missed that point.... I've disabled 'max' || 0 instances test because travis sometime has a ton of CPU.
Just fixed and added test: d5acb85

Once travis goes green I will hot publish this fix, expect it in one hour

Unitech · 2018-10-05T15:51:59Z

deployed:

$ npm install pm2@latest -g
$ pm2 update

curtisbelt · 2018-11-29T15:54:31Z

@Unitech Hello, this issue seems to be about Error: listen EADDRINUSE when using exec_mode: 'cluster'. @vinogradov-m pointed out a commit that referenced why this may be happening. There are also duplicate issues which reference this issue (#3996 and #3987).

But you seemed to have closed the issue after fixing an unrelated bug (setting instance to 'max' || 0).

Am I misunderstanding?

Note I am on 3.2.2 and have experienced the Error: listen EADDRINUSE bug -- and setting exec_mode: 'cluster_mode' fixed it. But I fear there is no issue tracking this problem, as they are all closed?

Thank you

cnscorpions · 2019-11-06T06:46:10Z

"So I defined exec_mod in my config file as 'cluster_mode'" not work

Unitech added the S: Pending Release label Oct 5, 2018

Unitech closed this as completed Oct 5, 2018

jdweeks mentioned this issue Oct 9, 2018

Patch for PM2 issues Optum/mockiato#79

Merged

kermitology mentioned this issue Oct 29, 2018

Restarted process in cluster mode can't bind to a port #3987

Closed

paz-raon mentioned this issue Nov 5, 2018

[v3.2.2] port already in use #3996

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

3.2.x cluster processes unable to bind to tcp listener port #3943

3.2.x cluster processes unable to bind to tcp listener port #3943

HughePaul commented Oct 5, 2018 •

edited

ufukd commented Oct 5, 2018

vinogradov-m commented Oct 5, 2018 •

edited

Unitech commented Oct 5, 2018

Unitech commented Oct 5, 2018

Unitech commented Oct 5, 2018

curtisbelt commented Nov 29, 2018

cnscorpions commented Nov 6, 2019

3.2.x cluster processes unable to bind to tcp listener port #3943

3.2.x cluster processes unable to bind to tcp listener port #3943

Comments

HughePaul commented Oct 5, 2018 • edited

What's going wrong?

How could we reproduce this issue?

Supporting information

ufukd commented Oct 5, 2018

vinogradov-m commented Oct 5, 2018 • edited

Unitech commented Oct 5, 2018

Unitech commented Oct 5, 2018

Unitech commented Oct 5, 2018

curtisbelt commented Nov 29, 2018

cnscorpions commented Nov 6, 2019

HughePaul commented Oct 5, 2018 •

edited

vinogradov-m commented Oct 5, 2018 •

edited