Add CLI Argument and XML Configuration to display PHPUnit warnings

[llaville]

Context

While I test for the first time https://github.com/ergebnis/phpunit-slow-test-detector on my project https://github.com/llaville/sarif-php-sdk, I've noticed a lot of warnings raised by PHPUnit when loading/bootstrapping an extension.

NB: I've added a PHP sleep command to display some lines with phpunit-slow-test-detector.

PHPUnit 11.5 run with phpunit-slow-test-detector 2.20

PHPUnit 11.5.34 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.6
Configuration: /home/devilbox/data/sarif-php-sdk/phpunit.xml.dist
Extension:     ergebnis/phpunit-slow-test-detector 2.20.0

................................................................. 65 / 72 ( 90%)
.......                                                           72 / 72 (100%)

Detected 36 tests where the duration exceeded the maximum duration.

 1. 00:01.012 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/runAutomationDetails" ('runAutomationDetails', Bartlett\Sarif\SarifLog Object (...))
 2. 00:01.009 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/suppression" ('suppression', Bartlett\Sarif\SarifLog Object (...))
 3. 00:01.008 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/sarifLog" ('sarifLog', Bartlett\Sarif\SarifLog Object (...))
 4. 00:01.008 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/tool" ('tool', Bartlett\Sarif\SarifLog Object (...))
 5. 00:01.005 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/versionControlDetails" ('versionControlDetails', Bartlett\Sarif\SarifLog Object (...))
 6. 00:01.004 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/message/plainText" ('message/plainText', Bartlett\Sarif\SarifLog Object (...))
 7. 00:01.004 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/stack" ('stack', Bartlett\Sarif\SarifLog Object (...))
 8. 00:01.004 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/specialLocations" ('specialLocations', Bartlett\Sarif\SarifLog Object (...))
 9. 00:01.003 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/message/stringLookup" ('message/stringLookup', Bartlett\Sarif\SarifLog Object (...))
10. 00:01.003 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/webRequest" ('webRequest', Bartlett\Sarif\SarifLog Object (...))
11. 00:01.003 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/translationMetadata" ('translationMetadata', Bartlett\Sarif\SarifLog Object (...))
12. 00:01.002 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/message/embeddedLinks" ('message/embeddedLinks', Bartlett\Sarif\SarifLog Object (...))
13. 00:01.002 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/message/formatted" ('message/formatted', Bartlett\Sarif\SarifLog Object (...))
14. 00:01.002 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/locationRelationship" ('locationRelationship', Bartlett\Sarif\SarifLog Object (...))
15. 00:01.002 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/result" ('result', Bartlett\Sarif\SarifLog Object (...))
16. 00:01.002 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/graphTraversal" ('graphTraversal', Bartlett\Sarif\SarifLog Object (...))
17. 00:01.002 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/exception" ('exception', Bartlett\Sarif\SarifLog Object (...))
18. 00:01.002 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/artifact" ('artifact', Bartlett\Sarif\SarifLog Object (...))
19. 00:01.002 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/reportingDescriptorReference" ('reportingDescriptorReference', Bartlett\Sarif\SarifLog Object (...))
20. 00:01.002 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/logicalLocation" ('logicalLocation', Bartlett\Sarif\SarifLog Object (...))
21. 00:01.002 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/conversion" ('conversion', Bartlett\Sarif\SarifLog Object (...))
22. 00:01.002 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/externalPropertyFileReferences" ('externalPropertyFileReferences', Bartlett\Sarif\SarifLog Object (...))
23. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/rectangle" ('rectangle', Bartlett\Sarif\SarifLog Object (...))
24. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/reportingDescriptor" ('reportingDescriptor', Bartlett\Sarif\SarifLog Object (...))
25. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/graph" ('graph', Bartlett\Sarif\SarifLog Object (...))
26. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/codeFlow" ('codeFlow', Bartlett\Sarif\SarifLog Object (...))
27. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/resultProvenance" ('resultProvenance', Bartlett\Sarif\SarifLog Object (...))
28. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/address" ('address', Bartlett\Sarif\SarifLog Object (...))
29. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/reportingDescriptorRelationship" ('reportingDescriptorRelationship', Bartlett\Sarif\SarifLog Object (...))
30. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/externalProperties" ('externalProperties', Bartlett\Sarif\SarifLog Object (...))
31. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/configurationOverride" ('configurationOverride', Bartlett\Sarif\SarifLog Object (...))
32. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/physicalLocation" ('physicalLocation', Bartlett\Sarif\SarifLog Object (...))
33. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/reportingConfiguration" ('reportingConfiguration', Bartlett\Sarif\SarifLog Object (...))
34. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/attachment" ('attachment', Bartlett\Sarif\SarifLog Object (...))
35. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/run" ('run', Bartlett\Sarif\SarifLog Object (...))
36. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Serializer\NativeJsonSerializerTest::testBuildReport with data set "examples/fix" ('fix', Bartlett\Sarif\SarifLog Object (...))

Time: 00:36.190, Memory: 42.00 MB

There were 33 PHPUnit test runner warnings:

1) /home/devilbox/data/box.phar is not an extension for PHPUnit

2) /home/devilbox/data/captainhook/phpunit-9.6.24.phar is not an extension for PHPUnit

3) /home/devilbox/data/easy-coding-standard/vendor/bin/phpstan.phar is not an extension for PHPUnit

4) /home/devilbox/data/easy-coding-standard/vendor/phpstan/phpstan/phpstan.phar is not an extension for PHPUnit

5) /home/devilbox/data/graph-plantuml-generator/graph-plantuml.phar is not an extension for PHPUnit

6) /home/devilbox/data/graph-plantuml-generator/vendor-bin/phplint/vendor/overtrue/phplint/resources/graph-uml.phar is not an extension for PHPUnit

7) /home/devilbox/data/graph-plantuml-generator/vendor-bin/phpstan/vendor/phpstan/phpstan/phpstan.phar is not an extension for PHPUnit

8) /home/devilbox/data/graph-plantuml-generator/vendor/bin/phpstan.phar is not an extension for PHPUnit

9) /home/devilbox/data/graph-uml.phar is not an extension for PHPUnit

10) /home/devilbox/data/graph-uml/vendor-bin/phplint/vendor/overtrue/phplint/resources/graph-uml.phar is not an extension for PHPUnit

11) /home/devilbox/data/graph-uml/vendor-bin/phpstan/vendor/phpstan/phpstan/phpstan.phar is not an extension for PHPUnit

12) /home/devilbox/data/graph-uml/vendor/bin/phpstan.phar is not an extension for PHPUnit

13) /home/devilbox/data/php-compatinfo-db/vendor-bin/phplint/vendor/overtrue/phplint/resources/graph-uml.phar is not an extension for PHPUnit

14) /home/devilbox/data/php-compatinfo-db/vendor-bin/phpstan/vendor/phpstan/phpstan/phpstan.phar is not an extension for PHPUnit

15) /home/devilbox/data/php-compatinfo-db/vendor/bin/phpstan.phar is not an extension for PHPUnit

16) /home/devilbox/data/php-cs-fixer.phar is not an extension for PHPUnit

17) /home/devilbox/data/phpunit-slow-test-detector/vendor/bin/phpstan.phar is not an extension for PHPUnit

18) /home/devilbox/data/phpunit-slow-test-detector/vendor/phpstan/phpstan/phpstan.phar is not an extension for PHPUnit

19) /home/devilbox/data/phpunit/tests/end-to-end/_files/phar-extension/tools/phpunit.d/phpunit-test-extension-1.0.0.phar is not compatible with PHPUnit 11.5

20) /home/devilbox/data/sarif-php-converters/vendor-bin/phplint/vendor/overtrue/phplint/resources/graph-uml.phar is not an extension for PHPUnit

21) /home/devilbox/data/sarif-php-converters/vendor-bin/phpstan/vendor/phpstan/phpstan/phpstan.phar is not an extension for PHPUnit

22) /home/devilbox/data/sarif-php-converters/vendor-bin/rector/vendor/phpstan/phpstan/phpstan.phar is not an extension for PHPUnit

23) /home/devilbox/data/sarif-php-converters/vendor/bin/phpstan.phar is not an extension for PHPUnit

24) /home/devilbox/data/sarif-php-sdk.phar is not an extension for PHPUnit

25) /home/devilbox/data/sarif-php-sdk/vendor-bin/phplint/vendor/overtrue/phplint/resources/graph-uml.phar is not an extension for PHPUnit

26) /home/devilbox/data/sarif-php-sdk/vendor-bin/phpstan/vendor/phpstan/phpstan/phpstan.phar is not an extension for PHPUnit

27) /home/devilbox/data/sarif-php-sdk/vendor/bin/phpstan.phar is not an extension for PHPUnit

28) /home/devilbox/data/umlwriter-4.3.phar is not an extension for PHPUnit

29) /home/devilbox/data/umlwriter.phar is not an extension for PHPUnit

30) /home/devilbox/data/umlwriter/bin/umlwriter.phar is not an extension for PHPUnit

31) /home/devilbox/data/umlwriter/vendor-bin/phplint/vendor/overtrue/phplint/resources/graph-uml.phar is not an extension for PHPUnit

32) /home/devilbox/data/umlwriter/vendor-bin/phpstan/vendor/phpstan/phpstan/phpstan.phar is not an extension for PHPUnit

33) /home/devilbox/data/umlwriter/vendor/bin/phpstan.phar is not an extension for PHPUnit

OK, but there were issues!
Tests: 72, Assertions: 360, PHPUnit Warnings: 33.

Local Dev context

Locally, on one of my test platform, I've a folder data/ where I put all PHAR distributions, and on sub-dir each project as its own directory.

Look like

data
├── app
├── assert
├── box.phar
├── captainhook
├── easy-coding-standard
├── graph-plantuml-generator
├── graph-uml
├── graph-uml.phar
├── harmony-package-template
├── megalinter
├── php-compatinfo-db
├── php-cs-fixer.phar
├── phpunit
├── phpunit-slow-test-detector
├── phpunit-slow-test-detector.phar
├── sarif-php-converters
├── sarif-php-sdk
├── sarif-php-sdk.phar
├── tmp
├── umlwriter
├── umlwriter-4.3.phar
└── umlwriter.phar

16 directories, 7 files

Feature Request

We can handle PHPUnit deprecations, warning and notice on test case and test suite

Execution:

  --fail-on-phpunit-deprecation        Signal failure using shell exit code when a PHPUnit deprecation was triggered
  --fail-on-phpunit-notice             Signal failure using shell exit code when a PHPUnit notice was triggered
  --fail-on-phpunit-warning            Signal failure using shell exit code when a PHPUnit warning was triggered

  --do-not-fail-on-phpunit-deprecation Do not signal failure using shell exit code when a PHPUnit deprecation was triggered
  --do-not-fail-on-phpunit-notice      Do not signal failure using shell exit code when a PHPUnit notice was triggered
  --do-not-fail-on-phpunit-warning     Do not signal failure using shell exit code when a PHPUnit warning was triggered

BUT we can ONLY display or NOT the PHPUnit deprecations and notice, no warning option exists yet

Reporting:

  --display-phpunit-deprecations       Display details for PHPUnit deprecations
  --display-phpunit-notices            Display details for PHPUnit notices

And of course for the XML configuration :

• failOnPhpunitDeprecation
• displayDetailsOnPhpunitDeprecations

But no setting/attribute for

• failOnPhpunitWarning (equivalent to --fail-on-phpunit-warning flag)

• failOnPhpunitNotice (equivalent to --fail-on-phpunit-notice flag)

• displayDetailsOnPhpunitWarnings

• displayDetailsOnPhpunitNotice

QUESTION Is it possible to have the same behaviour for PHPUnit warnings as it already exists for PHPUnit deprecations ? ONLY show on demand, and not always.

Reason: If you look on run output with PHPUnit 11.5 run with phpunit-slow-test-detector 2.20, I've 33 warnings displayed that is not revelant of success or failure and slow tests detection.

[llaville]

Performance summary

Without Extension

vendor/bin/phpunit --no-extension

Time: 00:00.029, Memory: 10.00 MB

With Extension

vendor/bin/phpunit

Time: 00:00.054, Memory: 42.00 MB

There were 33 PHPUnit test runner warnings:

Performance details from https://github.com/llaville/sarif-php-sdk (2.2.1) without PHP sleep code

With such patch applied to default phpunit.xml.dist (available on source code)

diff --git a/phpunit.xml.dist b/phpunit.xml.dist
index e93ef33..cbb20ce 100644
--- a/phpunit.xml.dist
+++ b/phpunit.xml.dist
@@ -4,6 +4,7 @@
     xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/11.5/phpunit.xsd"
     backupGlobals="false"
     bootstrap="vendor/autoload.php"
+    extensionsDirectory="../"
     colors="true"
     stopOnError="false"
     stopOnFailure="false"
@@ -13,6 +14,11 @@
     cacheDirectory=".phpunit.cache"
     backupStaticProperties="false"
 >
+    <extensions>
+        <bootstrap class="Ergebnis\PHPUnit\SlowTestDetector\Extension">
+            <parameter name="maximum-count" value="50"/>
+        </bootstrap>
+    </extensions>
     <coverage/>
     <testsuites>
         <testsuite name="Unit-Tests">

PHPUnit 11.5 run with phpunit-slow-test-detector 2.20 loaded

PHPUnit 11.5.34 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.6
Configuration: /home/devilbox/data/sarif-php-sdk/phpunit.xml.dist
Extension:     ergebnis/phpunit-slow-test-detector 2.20.0

................................................................. 65 / 72 ( 90%)
.......                                                           72 / 72 (100%)

Time: 00:00.054, Memory: 42.00 MB

There were 33 PHPUnit test runner warnings:

1) /home/devilbox/data/box.phar is not an extension for PHPUnit

2) /home/devilbox/data/captainhook/phpunit-9.6.24.phar is not an extension for PHPUnit

3) /home/devilbox/data/easy-coding-standard/vendor/bin/phpstan.phar is not an extension for PHPUnit

4) /home/devilbox/data/easy-coding-standard/vendor/phpstan/phpstan/phpstan.phar is not an extension for PHPUnit

5) /home/devilbox/data/graph-plantuml-generator/graph-plantuml.phar is not an extension for PHPUnit

6) /home/devilbox/data/graph-plantuml-generator/vendor-bin/phplint/vendor/overtrue/phplint/resources/graph-uml.phar is not an extension for PHPUnit

7) /home/devilbox/data/graph-plantuml-generator/vendor-bin/phpstan/vendor/phpstan/phpstan/phpstan.phar is not an extension for PHPUnit

8) /home/devilbox/data/graph-plantuml-generator/vendor/bin/phpstan.phar is not an extension for PHPUnit

9) /home/devilbox/data/graph-uml.phar is not an extension for PHPUnit

10) /home/devilbox/data/graph-uml/vendor-bin/phplint/vendor/overtrue/phplint/resources/graph-uml.phar is not an extension for PHPUnit

11) /home/devilbox/data/graph-uml/vendor-bin/phpstan/vendor/phpstan/phpstan/phpstan.phar is not an extension for PHPUnit

12) /home/devilbox/data/graph-uml/vendor/bin/phpstan.phar is not an extension for PHPUnit

13) /home/devilbox/data/php-compatinfo-db/vendor-bin/phplint/vendor/overtrue/phplint/resources/graph-uml.phar is not an extension for PHPUnit

14) /home/devilbox/data/php-compatinfo-db/vendor-bin/phpstan/vendor/phpstan/phpstan/phpstan.phar is not an extension for PHPUnit

15) /home/devilbox/data/php-compatinfo-db/vendor/bin/phpstan.phar is not an extension for PHPUnit

16) /home/devilbox/data/php-cs-fixer.phar is not an extension for PHPUnit

17) /home/devilbox/data/phpunit-slow-test-detector/vendor/bin/phpstan.phar is not an extension for PHPUnit

18) /home/devilbox/data/phpunit-slow-test-detector/vendor/phpstan/phpstan/phpstan.phar is not an extension for PHPUnit

19) /home/devilbox/data/phpunit/tests/end-to-end/_files/phar-extension/tools/phpunit.d/phpunit-test-extension-1.0.0.phar is not compatible with PHPUnit 11.5

20) /home/devilbox/data/sarif-php-converters/vendor-bin/phplint/vendor/overtrue/phplint/resources/graph-uml.phar is not an extension for PHPUnit

21) /home/devilbox/data/sarif-php-converters/vendor-bin/phpstan/vendor/phpstan/phpstan/phpstan.phar is not an extension for PHPUnit

22) /home/devilbox/data/sarif-php-converters/vendor-bin/rector/vendor/phpstan/phpstan/phpstan.phar is not an extension for PHPUnit

23) /home/devilbox/data/sarif-php-converters/vendor/bin/phpstan.phar is not an extension for PHPUnit

24) /home/devilbox/data/sarif-php-sdk.phar is not an extension for PHPUnit

25) /home/devilbox/data/sarif-php-sdk/vendor-bin/phplint/vendor/overtrue/phplint/resources/graph-uml.phar is not an extension for PHPUnit

26) /home/devilbox/data/sarif-php-sdk/vendor-bin/phpstan/vendor/phpstan/phpstan/phpstan.phar is not an extension for PHPUnit

27) /home/devilbox/data/sarif-php-sdk/vendor/bin/phpstan.phar is not an extension for PHPUnit

28) /home/devilbox/data/umlwriter-4.3.phar is not an extension for PHPUnit

29) /home/devilbox/data/umlwriter.phar is not an extension for PHPUnit

30) /home/devilbox/data/umlwriter/bin/umlwriter.phar is not an extension for PHPUnit

31) /home/devilbox/data/umlwriter/vendor-bin/phplint/vendor/overtrue/phplint/resources/graph-uml.phar is not an extension for PHPUnit

32) /home/devilbox/data/umlwriter/vendor-bin/phpstan/vendor/phpstan/phpstan/phpstan.phar is not an extension for PHPUnit

33) /home/devilbox/data/umlwriter/vendor/bin/phpstan.phar is not an extension for PHPUnit

OK, but there were issues!
Tests: 72, Assertions: 360, PHPUnit Warnings: 33.

Against extension not loaded

PHPUnit 11.5 run without phpunit-slow-test-detector 2.20 loaded

PHPUnit 11.5.34 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.6
Configuration: /home/devilbox/data/sarif-php-sdk/phpunit.xml.dist

................................................................. 65 / 72 ( 90%)
.......                                                           72 / 72 (100%)

Time: 00:00.029, Memory: 10.00 MB

OK (72 tests, 360 assertions)

[llaville]

If I remove the extensionDirectory attribute on PHPUnit config file

diff --git a/phpunit.xml.dist b/phpunit.xml.dist
index e93ef33..5583a06 100644
--- a/phpunit.xml.dist
+++ b/phpunit.xml.dist
@@ -13,6 +13,11 @@
     cacheDirectory=".phpunit.cache"
     backupStaticProperties="false"
 >
+    <extensions>
+        <bootstrap class="Ergebnis\PHPUnit\SlowTestDetector\Extension">
+            <parameter name="maximum-count" value="50"/>
+        </bootstrap>
+    </extensions>
     <coverage/>
     <testsuites>
         <testsuite name="Unit-Tests">

And run test suites, I got better good performance too

PHPUnit 11.5.34 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.6
Configuration: /home/devilbox/data/sarif-php-sdk/phpunit.xml.dist

................................................................. 65 / 72 ( 90%)
.......                                                           72 / 72 (100%)

Time: 00:00.029, Memory: 10.00 MB

There was 1 PHPUnit test runner warning:

1) Cannot bootstrap extension because class Ergebnis\PHPUnit\SlowTestDetector\Extension does not exist

OK, but there were issues!
Tests: 72, Assertions: 360, PHPUnit Warnings: 1.

So, if look-up PHAR compatible extensions found too much PHAR files, it may consume lot of time and memory !

[sebastianbergmann]

if look-up PHAR compatible extensions found too much PHAR files, it may consume lot of time and memory !

The directory configured using the extensionsDirectory attribute must only contain PHP Archive (PHAR) files containing PHPUnit extensions. Any file found that is not a PHPUnit extension PHAR will trigger a warning.

[sebastianbergmann]

You write that the XML configuration file does not have failOnPhpunitWarning, failOnPhpunitNotice, displayDetailsOnPhpunitWarnings, displayDetailsOnPhpunitNotice. But that is not entirely true.

failOnPhpunitWarning, failOnPhpunitNotice, and displayDetailsOnPhpunitNotices exist.

It is correct that displayDetailsOnPhpunitWarnings does not exist. This is by design as I do not believe that PHPUnit errors and warnings should be hidden.

[llaville]

Bootstrapping extension as a Composer package (see https://github.com/ergebnis/phpunit-slow-test-detector?tab=readme-ov-file#bootstrapping-the-extension-as-a-composer-package) rather than PHAR, gave better performance results, but I found another issue : I'll open a separated ticket

PHPUnit 11.5.34 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.6
Configuration: /home/devilbox/data/sarif-php-sdk/phpunit.xml.dist

................................................................. 65 / 72 ( 90%)
.......                                                           72 / 72 (100%)

Detected 36 tests where the duration exceeded the maximum duration.

 1. 00:01.010 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/message/embeddedLinks" ('message/embeddedLinks', Bartlett\Sarif\SarifLog Object (...))
 2. 00:01.008 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/logicalLocation" ('logicalLocation', Bartlett\Sarif\SarifLog Object (...))
 3. 00:01.005 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/rectangle" ('rectangle', Bartlett\Sarif\SarifLog Object (...))
 4. 00:01.005 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/reportingDescriptor" ('reportingDescriptor', Bartlett\Sarif\SarifLog Object (...))
 5. 00:01.002 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/physicalLocation" ('physicalLocation', Bartlett\Sarif\SarifLog Object (...))
 6. 00:01.002 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/graphTraversal" ('graphTraversal', Bartlett\Sarif\SarifLog Object (...))
 7. 00:01.002 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/reportingConfiguration" ('reportingConfiguration', Bartlett\Sarif\SarifLog Object (...))
 8. 00:01.002 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/result" ('result', Bartlett\Sarif\SarifLog Object (...))
 9. 00:01.002 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/specialLocations" ('specialLocations', Bartlett\Sarif\SarifLog Object (...))
10. 00:01.002 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/resultProvenance" ('resultProvenance', Bartlett\Sarif\SarifLog Object (...))
11. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/attachment" ('attachment', Bartlett\Sarif\SarifLog Object (...))
12. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/reportingDescriptorReference" ('reportingDescriptorReference', Bartlett\Sarif\SarifLog Object (...))
13. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/translationMetadata" ('translationMetadata', Bartlett\Sarif\SarifLog Object (...))
14. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/stack" ('stack', Bartlett\Sarif\SarifLog Object (...))
15. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/configurationOverride" ('configurationOverride', Bartlett\Sarif\SarifLog Object (...))
16. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/message/stringLookup" ('message/stringLookup', Bartlett\Sarif\SarifLog Object (...))
17. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/codeFlow" ('codeFlow', Bartlett\Sarif\SarifLog Object (...))
18. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/locationRelationship" ('locationRelationship', Bartlett\Sarif\SarifLog Object (...))
19. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/webRequest" ('webRequest', Bartlett\Sarif\SarifLog Object (...))
20. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/externalPropertyFileReferences" ('externalPropertyFileReferences', Bartlett\Sarif\SarifLog Object (...))
21. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/versionControlDetails" ('versionControlDetails', Bartlett\Sarif\SarifLog Object (...))
22. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/suppression" ('suppression', Bartlett\Sarif\SarifLog Object (...))
23. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/run" ('run', Bartlett\Sarif\SarifLog Object (...))
24. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/fix" ('fix', Bartlett\Sarif\SarifLog Object (...))
25. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/runAutomationDetails" ('runAutomationDetails', Bartlett\Sarif\SarifLog Object (...))
26. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/message/formatted" ('message/formatted', Bartlett\Sarif\SarifLog Object (...))
27. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/externalProperties" ('externalProperties', Bartlett\Sarif\SarifLog Object (...))
28. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/reportingDescriptorRelationship" ('reportingDescriptorRelationship', Bartlett\Sarif\SarifLog Object (...))
29. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/message/plainText" ('message/plainText', Bartlett\Sarif\SarifLog Object (...))
30. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/exception" ('exception', Bartlett\Sarif\SarifLog Object (...))
31. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/artifact" ('artifact', Bartlett\Sarif\SarifLog Object (...))
32. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/conversion" ('conversion', Bartlett\Sarif\SarifLog Object (...))
33. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/tool" ('tool', Bartlett\Sarif\SarifLog Object (...))
34. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/address" ('address', Bartlett\Sarif\SarifLog Object (...))
35. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/graph" ('graph', Bartlett\Sarif\SarifLog Object (...))
36. 00:01.001 (00:00.500) Bartlett\Sarif\Tests\unit\Builder\BuilderFactoryTest::testBuilderFactory with data set "examples/builder/sarifLog" ('sarifLog', Bartlett\Sarif\SarifLog Object (...))

Time: 00:36.154, Memory: 10.00 MB

OK (72 tests, 360 assertions)

[llaville]

if look-up PHAR compatible extensions found too much PHAR files, it may consume lot of time and memory !

The directory configured using the extensionsDirectory attribute must only contain PHP Archive (PHAR) files containing PHPUnit extensions. Any file found that is not a PHPUnit extension PHAR will trigger a warning.

Finally Composer Package bootstrapping is enough with better performance (for me), at a little details (bug) explained by report #6334