pcntl_fork children are utilizing just a single core on FreeBSD 13.3+

[instaproofs]

Description

Using PHP 8.2.18, I have a script that processes a series of images, performing various conversions and resizing tasks. To make the most of the server's multiple cores, I use the pcntl_fork() function to create child processes that can simultaneously handle different images. This means instead of processing images sequentially, each image can be processed concurrently on separate cores.

For instance, if I have 10 images to process and each takes 3 seconds individually, without parallel processing, it would take a total of 30 seconds. However, with parallel processing, all 10 images can finish processing simultaneously in just 3 seconds.

This approach has been effective until we updated to FreeBSD 13.3. After the update, the forked processes no longer distribute across different cores; instead, they all run on a single core. Consequently, if I have 10 forked processes running, each is constrained to using only 10% of a single core, resulting in a 10-fold increase in processing time.

We've conducted tests with FreeBSD versions ranging from 9.x up to 13.2-RELEASE-p11 and found that the issue doesn't occur. However, when we update to FreeBSD 13.3, the problem occurs every time.

We've tested across several different PHP versions, from 8.1x up to 8.3x, and they all yield the same results.

This may sound like a FreeBSD issue, but I am at a loss and exploring all avenues for it.

Anyone have any ideas on what might be causing this issue, or any suggestions for potentially resolving it?

Sample code for forking the processes...

<?PHP
$fullLocalImagePath = "test.jpg";
$forkedProcessIds = array();
for($i = 0; $i < 5; $i++) {
  $pid = pcntl_fork();// Create a child process
  if($pid == -1) {
    exit(-1); // Could not fork the process
  } else if($pid) {
    $forkedProcessIds[] = $pid; // Keep track of the child IDs
  } else {
    $finalImagePath = "test_result_".$i.".jpg";

    // Load the image using Imagick
    $imagick = new Imagick();
    $imagick->readImage($fullLocalImagePath);
    echo " → Finished reading image $i into Imagick.\n";
    $imagick->setImageCompressionQuality(88);
    $imagick->resizeImage(4800, 4800, imagick::FILTER_LANCZOS, .9, false);
    $imagick->writeImage($finalImagePath);
    echo " → → Finished resizing and saving image $i into Imagick.\n";
    $imagick->clear();

    exit(0); // Exit the child process after processing the image
  }
}

// Wait for the forked processes to finish
while ($childPID = pcntl_waitpid(0, $status)) {
  if ($childPID == -1) {
    // No child processes left to wait for
    break;
  }

  echo " → → → Child process $childPID has finished.\n";

  // Handle the exit status based on the child process PID
  if (in_array($childPID, $forkedProcessIds)) {
    // Remove the child process ID from the tracking array
    $forkedProcessIds = array_diff($forkedProcessIds, array($childPID));
  }
}
?>

PHP Version

PHP 8.2.18

Operating System

FreeBSD 13.3

** Edited to add a more complete code example.

[devnexen]

The next PHP major release will allow to have more control over processes being bound to specific cores/cpus (via pcntl). For now, maybe there is a change in term of cpu topology policy ? you may look at the following value

sysctl kern.sched.topology_spec

and see if changing it has an impact. But I may have a better look later.

[instaproofs]

We have narrowed this down to it being somehow related to Imagick. If we run the same code but do anything besides load Imagick in the forked processes, it works as expected and spreads the forked processes to all of the available CPUs. For example, if the images are resized using the GD library instead, it works properly. Or if we just read from /dev/random, it works properly as well.

The same script, using Imagick, works properly on everything below FreeBSD 13.3, so it's still a mystery as to why it is not functioning correctly after updating. If anyone has suggestions, we are still all-ears. Thank you!

[devnexen]

quick question: Do you know if your imagemagick port is (and/or was) compiled with openmp support ?

[kmpanilla]

quick question: Do you know if your imagemagick port is (and/or was) compiled with openmp support ?

Yes, our ImageMagick7 is compiled WITH OpenMP support. Funny enough, with OpenMP disabled, forked processes use all the CPU cores as expected. With OpenMP compiled in, the results are limited to one CPU.

[devnexen]

Good to know :-). Anyhow, not a php issue.