On PM2 cluster mode non primary instances exit on start

[jimboolio]

Preflight Checklist

• I have completed all Troubleshooting Steps.
• I'm on the latest version of Directus.
• There's no other issue that already describes my problem.

Describe the Bug

I'm trying to set up directus to run on PM2 load balanced cluster which would utilize all available cpu cores, but additional instances of Directus exit with App [directus:1] exited with code [1] via signal [SIGINT]

There are no error messages on the console so I guess this could be as designed, if this is the case, it could be beneficial to include this information in the documentation

To Reproduce

My ecosystem.config.js:

module.exports = {
  apps : [{
    name   : "directus",
    instances : "2",
    exec_mode : "cluster",
    script : "npx",
    args : "directus start",
    autorestart : false
  }]
}

Caching is disabled on .env:

CACHE_ENABLED=false
CACHE_SCHEMA=false
CACHE_PERMISSIONS=false
RATE_LIMITER_ENABLED=false

Running pm2 restart ecosystem.config.js --attach outputs the following:

[PM2] Applying action restartProcessId on app [directus](ids: [ 0, 1 ])
[PM2] [directus](0) ✓
[PM2] [directus](1) ✓
┌─────┬─────────────┬─────────────┬─────────┬─────────┬──────────┬────────┬──────┬───────────┬──────────┬──────────┬──────────┬──────────┐
│ id  │ name        │ namespace   │ version │ mode    │ pid      │ uptime │ ↺    │ status    │ cpu      │ mem      │ user     │ watching │
├─────┼─────────────┼─────────────┼─────────┼─────────┼──────────┼────────┼──────┼───────────┼──────────┼──────────┼──────────┼──────────┤
│ 0   │ directus    │ default     │ N/A     │ cluster │ 307      │ 0s     │ 1    │ online    │ 0%       │ 35.2mb   │ adm… │ disabled │
│ 1   │ directus    │ default     │ N/A     │ cluster │ 314      │ 0s     │ 0    │ online    │ 0%       │ 30.0mb   │ adm… │ disabled │
└─────┴─────────────┴─────────────┴─────────┴─────────┴──────────┴────────┴──────┴───────────┴──────────┴──────────┴──────────┴──────────┘
[PM2][INFO] Log streaming apps id: 0 1, exit with Ctrl-C or will exit in 10secs
PM2      | App name:directus id:1 disconnected
PM2      | App [directus:1] exited with code [1] via signal [SIGINT]

And when running pm2 listall but the first directus processes are stopped

Expected behaviour

Directus would run in as many processes as defined in PM2 ecosystem.config.js

Actual behaviour

Only the proces with PM2 id 0 will run, other processes exit

Errors Shown

No errors, it exits cleanly

What version of Directus are you using?

9.4.1

What version of Node.js are you using?

16.13.1

What database are you using?

Postgres 14.1

What browser are you using?

Firefox

What operating system are you using?

Linux 4.15.0-163-generic (Ubuntu 18.04.6 LTS)

How are you deploying Directus?

PM2 non containered

[eikaramba]

maybe try to use a script like this. It helped me to actually get the console output
server.js

const { spawn } = require('child_process');
const ls = spawn('npx',['directus','start']);

ls.stdout.on('data', (data) => {
  console.log(data.toString());
});

ls.stderr.on('data', (data) => {
  console.error(data.toString());
});

ls.on('close', (code) => {
  console.log(`server stopped ${code}`);
});

{
  "apps" : [{
    "name"        : "backend",
    "cwd"         : "/xxxxx",
    "script"      : "./server.js",
    "ignoreWatch" : ["[\\/\\\\]\\./", "node_modules"],
    "watch"       : "false",
    "env": {
        "CONFIG_PATH":"/xxxxx/.env.production",
        "NODE_ENV": "production"
    },
    "exec_mode"  : "cluster_mode",
    "error" : "/xxxxx/logs/backend-err.log",
    "output"   : "/xxxx/logs/backend-out.log",
    "port":1338,
    "time": true
  },

[rijkvanzanten]

@jimboolio is it confirmed that it was Directus that stopped running? The part of the message ... via signal [SIGINT] indicates to me that Directus received the SIGINT command, causing it to stop 🤔

[jimboolio]

This is most likely the issue. With the server.js script from @eikaramba I managed to find out that the other instances get terminated because the directus port is already in use

It looks like that in the pm2 ecosystem.config.js should point to an actual directus startup script/file, not npm or npx - I will update/reply here if I manage to figure that out...