Support: Split frontend/background workers

[pburgisser]

Support Questions

Hi folks,

In a fork of the misp-docker from CoolAcid, I'm trying the split the workers from the frontend.
From version 2.4.151, using the SimpleBackgroundJobs configuration, would it possible to run the workers away from the rest of the frontend ?

I already tried but I'm getting PHP errors (see below)
Thanks for your help

MISP version

2.4.161

Operating System

Ubuntu

Operating System version

bullseys

PHP version

7.4

Browser

Chrome

Browser version

No response

Relevant log output

2022-09-26 20:44:18 Error: [TypeError] Return value of Supervisor\Process::offsetGet() must be an instance of Supervisor\mixed, string returned
Request URL: /servers/serverSettings
Stack Trace:
#0 /var/www/MISP/app/Lib/Tools/BackgroundJobsTool.php(294): Supervisor\Process->offsetGet()
#1 /var/www/MISP/app/Model/Server.php(7565): BackgroundJobsTool->getWorkers()
#2 /var/www/MISP/app/Model/Server.php(3559): Server->getWorkers()
#3 /var/www/MISP/app/Controller/ServersController.php(1204): Server->workerDiagnostics()
#4 [internal function]: ServersController->serverSettings()
#5 /var/www/MISP/app/Lib/cakephp/lib/Cake/Controller/Controller.php(499): ReflectionMethod->invokeArgs()
#6 /var/www/MISP/app/Lib/cakephp/lib/Cake/Routing/Dispatcher.php(193): Controller->invokeAction()
#7 /var/www/MISP/app/Lib/cakephp/lib/Cake/Routing/Dispatcher.php(167): Dispatcher->_invoke()
#8 /var/www/MISP/app/webroot/index.php(99): Dispatcher->dispatch()
#9 {main}

Extra attachments

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[pburgisser]

I have the same issue when running both the frontend and the workers within the same container.

[righel]

Yes, it should be possible to move the workers to a backend.

The error you are getting is strange, could it be that you have installed supervisor:^5.0 ? version 5 is for PHP 8, won't work on PHP 7.4. It should be supervisorphp/supervisor:^4.0 (< 5.0)

Also, for using the SimpleBackgroundJobs check on the coolacid/docker-misp check this repo: coolacid/docker-misp#184, @ostefano did a great job for properly configuring it, unfortunately it hasn't been merged yet

[pburgisser]

The supervisor's version was the culprit, thanks !

Now, on the frontend, can list the workers, but it seems unhappy (see screenshot)

When I look at my worker container, I can see that all workers where started as www-data:

root@misp-blue-workers-5dcdf9b457-94xpx:/var/www/MISP# ps -u www-data -f
UID        PID  PPID  C STIME TTY          TIME CMD
www-data    53    52  0 12:03 ?        00:00:00 php -q /var/www/MISP/app/Console/cake.php -working /var/www/MISP/app start_worker default
www-data    54    52  0 12:03 ?        00:00:00 php -q /var/www/MISP/app/Console/cake.php -working /var/www/MISP/app start_worker default
www-data    56    52  0 12:03 ?        00:00:00 php -q /var/www/MISP/app/Console/cake.php -working /var/www/MISP/app start_worker default
www-data    57    52  0 12:03 ?        00:00:00 php -q /var/www/MISP/app/Console/cake.php -working /var/www/MISP/app start_worker default
www-data    58    52  0 12:03 ?        00:00:00 php -q /var/www/MISP/app/Console/cake.php -working /var/www/MISP/app start_worker default
www-data    61    52  0 12:03 ?        00:00:00 php -q /var/www/MISP/app/Console/cake.php -working /var/www/MISP/app start_worker email
www-data    64    52  0 12:03 ?        00:00:00 php -q /var/www/MISP/app/Console/cake.php -working /var/www/MISP/app start_worker email
www-data    67    52  0 12:03 ?        00:00:00 php -q /var/www/MISP/app/Console/cake.php -working /var/www/MISP/app start_worker email
www-data    72    52  0 12:03 ?        00:00:00 php -q /var/www/MISP/app/Console/cake.php -working /var/www/MISP/app start_worker email
www-data    75    52  0 12:03 ?        00:00:00 php -q /var/www/MISP/app/Console/cake.php -working /var/www/MISP/app start_worker email
www-data    86    52  0 12:03 ?        00:00:00 php -q /var/www/MISP/app/Console/cake.php -working /var/www/MISP/app start_worker cache
www-data    97    52  0 12:03 ?        00:00:00 php -q /var/www/MISP/app/Console/cake.php -working /var/www/MISP/app start_worker cache
www-data   109    52  0 12:03 ?        00:00:00 php -q /var/www/MISP/app/Console/cake.php -working /var/www/MISP/app start_worker cache
www-data   113    52  0 12:03 ?        00:00:00 php -q /var/www/MISP/app/Console/cake.php -working /var/www/MISP/app start_worker cache
www-data   117    52  0 12:03 ?        00:00:00 php -q /var/www/MISP/app/Console/cake.php -working /var/www/MISP/app start_worker cache
www-data   121    52  0 12:03 ?        00:00:00 php -q /var/www/MISP/app/Console/cake.php -working /var/www/MISP/app start_worker prio
www-data   130    52  0 12:03 ?        00:00:00 php -q /var/www/MISP/app/Console/cake.php -working /var/www/MISP/app start_worker prio
www-data   134    52  0 12:03 ?        00:00:00 php -q /var/www/MISP/app/Console/cake.php -working /var/www/MISP/app start_worker prio
www-data   138    52  0 12:03 ?        00:00:00 php -q /var/www/MISP/app/Console/cake.php -working /var/www/MISP/app start_worker prio
www-data   144    52  0 12:03 ?        00:00:00 php -q /var/www/MISP/app/Console/cake.php -working /var/www/MISP/app start_worker prio
www-data   150    52  0 12:03 ?        00:00:00 php -q /var/www/MISP/app/Console/cake.php -working /var/www/MISP/app start_worker update

If I look at the object returned by Supervisor, there isn't any user shown in the object:

array(21) { [0]=> object(Supervisor\Process)#154 (1) { ["payload":protected]=> array(14) { ["name"]=> string(8) "cache_00" ["group"]=> string(12) "misp-workers" ["start"]=> int(1664280239) ["stop"]=> int(0) ["now"]=> int(1664281330) ["state"]=> int(20) ["statename"]=> string(7) "RUNNING" ["spawnerr"]=> string(0) "" ["exitstatus"]=> int(0) ["logfile"]=> string(43) "/var/www/MISP/app/tmp/logs/misp-workers.log" ["stdout_logfile"]=> string(43) "/var/www/MISP/app/tmp/logs/misp-workers.log" ["stderr_logfile"]=> string(50) "/var/www/MISP/app/tmp/logs/misp-workers-errors.log" ["pid"]=> int(86) ["description"]=> string(22) "pid 86, uptime 0:18:11" } } [1]=> object(Supervisor\Process)#151 (1) { ["payload":protected]=> array(14) { ["name"]=> string(8) "cache_01" ["group"]=> string(12) "misp-workers" ["start"]=> int(1664280239) ["stop"]=> int(0) ["now"]=> int(1664281330) ["state"]=> int(20) ["statename"]=> string(7) "RUNNING" ["spawnerr"]=> string(0) "" ["exitstatus"]=> int(0) ["logfile"]=> string(43) "/var/www/MISP/app/tmp/logs/misp-workers.log" ["stdout_logfile"]=> string(43) "/var/www/MISP/app/tmp/logs/misp-workers.log" ["stderr_logfile"]=> string(50) "/var/www/MISP/app/tmp/logs/misp-workers-errors.log" ["pid"]=> int(97) ["description"]=> string(22) "pid 97, uptime 0:18:11" }

However, the jobs seem happy though:

Thanks for your support!

[righel]

The user of the worker processes comes from here: https://github.com/MISP/MISP/blob/2.4/app/Lib/Tools/BackgroundJobsTool.php#L720

Could you check/debug what is that function returning?

[pburgisser]

If I look at the code you gave, I think it will never work as it's trying to gather information from the worker's process on the same host whereas my worker is working somewhere else. What do you think ?

[righel]

You are correct, the PID will not exist or will not match the correct process, the code should be refactored to properly diagnose workers running in a different host.

[pburgisser]

I was correct:

2022-09-27 13:14:26 Error: PID: 86
2022-09-27 13:14:26 Error: PID 86 not found
2022-09-27 13:14:26 Error: PID: 97
2022-09-27 13:14:26 Error: PID 97 not found
2022-09-27 13:14:26 Error: PID: 109
2022-09-27 13:14:26 Error: PID 109 not found
2022-09-27 13:14:26 Error: PID: 113
2022-09-27 13:14:26 Error: PID 113 not found
2022-09-27 13:14:26 Error: PID: 117
2022-09-27 13:14:26 Error: PID 117 not found
2022-09-27 13:14:26 Error: PID: 53
2022-09-27 13:14:26 Error: PID 53 not found
2022-09-27 13:14:26 Error: PID: 54
2022-09-27 13:14:26 Error: PID 54 not found
2022-09-27 13:14:26 Error: PID: 56
2022-09-27 13:14:26 Error: PID 56 not found
2022-09-27 13:14:26 Error: PID: 57
2022-09-27 13:14:26 Error: PID 57 not found
2022-09-27 13:14:26 Error: PID: 58
2022-09-27 13:14:26 Error: PID 58 not found
2022-09-27 13:14:26 Error: PID: 61
2022-09-27 13:14:26 Error: PID 61 not found
2022-09-27 13:14:26 Error: PID: 64
2022-09-27 13:14:26 Error: PID 64 not found
2022-09-27 13:14:26 Error: PID: 67
2022-09-27 13:14:26 Error: PID 67 not found
2022-09-27 13:14:26 Error: PID: 72
2022-09-27 13:14:26 Error: PID 72 not found
2022-09-27 13:14:26 Error: PID: 75
2022-09-27 13:14:26 Error: For PID 75, there is a /proc/75/status
2022-09-27 13:14:26 Error: PID: 121
2022-09-27 13:14:26 Error: PID 121 not found
2022-09-27 13:14:26 Error: PID: 130
2022-09-27 13:14:26 Error: PID 130 not found
2022-09-27 13:14:26 Error: PID: 134
2022-09-27 13:14:26 Error: PID 134 not found
2022-09-27 13:14:26 Error: PID: 138
2022-09-27 13:14:26 Error: PID 138 not found
2022-09-27 13:14:26 Error: PID: 144
2022-09-27 13:14:26 Error: PID 144 not found
2022-09-27 13:14:26 Error: PID: 150
2022-09-27 13:14:26 Error: PID 150 not found

For this "hacked" code:

    private function processUser(int $pid)
    {
        CakeLog::error("PID: {$pid}");
        if (function_exists('posix_getpwuid') && file_exists("/proc/$pid/status")) {
            CakeLog::error("For PID {$pid}, there is a /proc/{$pid}/status");
            $content = file_get_contents("/proc/$pid/status");
            preg_match("/Uid:\t([0-9]+)\t([0-9]+)/", $content, $matches);
            return posix_getpwuid((int)$matches[2])['name'];
        } else {
           $foo = shell_exec(sprintf("ps -o uname='' -p %s", $pid));
           if ($foo == ""){
            CakeLog::error("PID {$pid} not found");
           }else{
                   CakeLog::error("PID {$pid} found");
           }

            return trim($foo ?? '');
        }
    }