PHPUnit9 @runInSeparateProcess is very slow in group execution

[naletups]

Q	A
PHPUnit version	9.3.latest - 9.5.2
xdebug	3.0.2
PHP version	7.3.21, 8.0.2
Installation Method	PHAR

Summary

I have a set of HTTP response management classes which I want to test. Some of them require @runInSeparateProcess. This is slightly slower than executing it directly - which is obvious.

Doing a single execution of that particular testcase, it takes ~0.4sec execution time. Executing the entire testcase file, for which roughly 50% of the testcases have @runInSeparateProcess set, results in an expected duration (~6sec). The command my IDE (VSCode with docker) executes for that is phpunit -c /workspaces/backend/ci/phpunit.xml --coverage-text --coverage-html ./logs '/workspaces/backend/tests/WebTests'.

Current behavior

The problem is if I execute all testcases back-to-back, using my XML config file, the testcases with the @runInSeparateProcess label suddenly take 5-10sec per single testcase to be executed. All other testcases take normal time. This slowdown can be observed with PHP7.3.x and PHP8.0.x.

The command of the IDE for all testcases is phpunit --configuration ./ci/phpunit.xml --coverage-html ./logs with the following XML config file.

phpunit.xml

<?xml version="1.0"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" bootstrap="../tests/autoload.php" beStrictAboutChangesToGlobalState="true" colors="false" xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/9.3/phpunit.xsd">
  <coverage processUncoveredFiles="true">
    <include>
      <directory suffix=".php">../src</directory>
    </include>
    <exclude>
      <file>../src/api.php</file>
      <file>../src/login.php</file>
      <directory>../src/libs</directory>
      <directory>../src/shared</directory>
    </exclude>
    <report>
      <text outputFile="php://stdout" showUncoveredFiles="false"/>
    </report>
  </coverage>
  <extensions>
    <extension class="Test\DbEnv"/>
  </extensions>
  <testsuites>
    <testsuite name="Unit">
      <directory suffix="Test.php">../tests</directory>
      <exclude>../tests/IntegrationTests</exclude>
    </testsuite>
  </testsuites>
  <logging/>
</phpunit>

Expected behavior

The expected behavior would be no massive slow down. Using PHPUnit 8.5.x with PHP 7.3.x (and probably also 7.4.x), there is no massive slowdown for these testcases.

Are there any pointers which I can test? I tried already quite some combinations of PHPUnit and PHP. The only thing I left untouched so far is xdebug.

[naletups]

And maybe (or maybe not) related: it seems, I cannot properly pass another php.ini file anymore.
The test scripts within docker call phpunit with a special php.ini file (that is also used for composer) like this php -c /usr/local/etc/php/composer.ini /usr/local/bin/phpunit --configuration ./ci/phpunit.xml --coverage-html ./logs/coverage/

However, it seems that PHPUnit9 now uses the default php.ini, which in my case has disabled_functions = chdir, which breaks some testcases (funny enough exactly those, that also turned out to be slow, see above). Making adjustments to the php.ini and using the above command, immediately make all testcases pass as expected (but those with @runInSeparateProcess again very slow).

[naletups]

After some more investigations, I think I tracked it down a bit: could it be that this is a RAM availability issue?!
I have 8GB available for these 2k++ tests. I already had to increase the PHP memory limits since the defaults were not enough for general operation. Could it be that in particular the @runInSeparateProcess need considerably more memory?