Deprecations not reported in the summary when running all unit tests

[abodnar]

Q	A
PHPUnit version	10.5.16
PHP version	8.2.17
Installation Method	Composer

Summary

While upgrading my application from PHP 8.1 to 8.2, I was validating that the code and unit tests had no issues and no deprecation issues. When I ran the unit tests, I saw output that there was a deprecation, but the summary just reported OK. If I ran the individual unit test, the summary reports the deprecation.

Current behavior

When I ran the unit tests, I saw this in the output:

Deprecated: Creation of dynamic property Tests\Unit\Dashboards\PrefillDateArrayTest::$defaultArrayItem is deprecated in /app/tests/Unit/Dashboards/PrefillDateArrayTest.php on line 15

But the summary was OK (97 tests, 214 assertions).

This even happens if I use --display-deprecations, it does not report the deprecations in the summary.

If I run the individual unit test, I got this:

1 test triggered 1 PHP deprecation:

1) /app/tests/Unit/Dashboards/PrefillDateArrayTest.php:15
Creation of dynamic property Tests\Unit\Dashboards\PrefillDateArrayTest::$defaultArrayItem is deprecated

Triggered by:

* Tests\Unit\Dashboards\PrefillDateArrayTest::testValidDateRangeAndFirstDayOfMonthByDay
  /app/tests/Unit/Dashboards/PrefillDateArrayTest.php:21

OK, but there were issues!
Tests: 1, Assertions: 3, Deprecations: 1.

How to reproduce

Have the phpunit.xml set to report deprecations(my configuration below) and have a unit test that triggers a depcreation warning(like using a dynamic property on a class that isn't defined). Run the unit tests and see the warning, but not mentioned in the summary. Then run just the unit test and see that the summary reports it.

Expected behavior

Deprecations are reported if the different deprecation settings in phpunit.xml are specified as true or --display-deprecations is used on the command line in the summary.

phpunit.xml:

<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd" bootstrap="vendor/autoload.php" colors="true" cacheDirectory=".phpunit.cache" stopOnDeprecation="true" failOnDeprecation="true" displayDetailsOnTestsThatTriggerDeprecations="true">
  <testsuites>
    <testsuite name="Unit">
      <directory suffix="Test.php">./tests/Unit</directory>
    </testsuite>
  </testsuites>
  <coverage/>
  <php>
    <env name="APP_ENV" value="testing"/>
    <env name="BCRYPT_ROUNDS" value="4"/>
    <env name="CACHE_DRIVER" value="array"/>
    <env name="DB_DATABASE" value="testing"/>
    <env name="MAIL_MAILER" value="array"/>
    <env name="QUEUE_CONNECTION" value="sync"/>
    <env name="SESSION_DRIVER" value="array"/>
    <env name="TELESCOPE_ENABLED" value="false"/>
  </php>
  <source>
    <include>
      <directory suffix=".php">./app</directory>
    </include>
  </source>
</phpunit>

Composer info:

brick/math                         0.11.0
carbonphp/carbon-doctrine-types    2.1.0
defuse/php-encryption              v2.4.0
dflydev/dot-access-data            v3.0.2
doctrine/cache                     2.2.0
doctrine/dbal                      3.8.3
doctrine/deprecations              1.1.3
doctrine/event-manager             2.0.0
doctrine/inflector                 2.0.10
doctrine/lexer                     3.0.1
dragonmantank/cron-expression      v3.3.3
egulias/email-validator            4.0.2
fakerphp/faker                     v1.23.1
fastly/fastly                      v5.1.0
fgt/thundercat-models              dev-ATLAS-113-php-8.2-upgrade 736a901
filp/whoops                        2.15.4
firebase/php-jwt                   v6.10.0
fruitcake/php-cors                 v1.3.0
graham-campbell/result-type        v1.1.2
guzzlehttp/guzzle                  7.8.1
guzzlehttp/promises                2.0.2
guzzlehttp/psr7                    2.6.2
guzzlehttp/uri-template            v1.0.3
hamcrest/hamcrest-php              v2.0.1
kitloong/laravel-app-logger        v1.2.2
laravel/framework                  v10.48.4
laravel/passport                   v11.10.6
laravel/pint                       v1.15.1
laravel/prompts                    v0.1.17
laravel/sail                       v1.20.2
laravel/serializable-closure       v1.3.3
laravel/tinker                     v2.9.0
lcobucci/clock                     3.2.0
lcobucci/jwt                       5.2.0
league/commonmark                  2.4.2
league/config                      v1.2.0
league/event                       2.2.0
league/flysystem                   3.26.0
league/flysystem-local             3.25.1
league/mime-type-detection         1.15.0
league/oauth2-server               8.5.4
league/uri                         7.4.1
league/uri-interfaces              7.4.1
mockery/mockery                    1.6.11
monolog/monolog                    3.5.0
myclabs/deep-copy                  1.11.1
nesbot/carbon                      2.72.3
nette/schema                       v1.3.0
nette/utils                        v4.0.4
nikic/php-parser                   v5.0.2
nunomaduro/collision               v7.10.0
nunomaduro/termwind                v1.15.1
nyholm/psr7                        1.8.1
paragonie/constant_time_encoding   v2.6.3
paragonie/random_compat            v9.99.100
phar-io/manifest                   2.0.4
phar-io/version                    3.2.1
phpoption/phpoption                1.9.2
phpseclib/phpseclib                3.0.37
phpunit/php-code-coverage          10.1.14
phpunit/php-file-iterator          4.1.0
phpunit/php-invoker                4.0.0
phpunit/php-text-template          3.0.1
phpunit/php-timer                  6.0.0
phpunit/phpunit                    10.5.16
psr/cache                          3.0.0
psr/clock                          1.0.0
psr/container                      2.0.2
psr/event-dispatcher               1.0.0
psr/http-client                    1.0.3
psr/http-factory                   1.0.2
psr/http-message                   2.0
psr/log                            3.0.0
psr/simple-cache                   3.0.0
psy/psysh                          v0.12.3
rakutentech/laravel-request-docs   v2.34
ralouphie/getallheaders            3.0.3
ramsey/collection                  2.0.0
ramsey/uuid                        4.7.5
roave/security-advisories          dev-latest 8f58125
sebastian/cli-parser               2.0.1
sebastian/code-unit                2.0.0
sebastian/code-unit-reverse-lookup 3.0.0
sebastian/comparator               5.0.1
sebastian/complexity               3.2.0
sebastian/diff                     5.1.1
sebastian/environment              6.1.0
sebastian/exporter                 5.1.2
sebastian/global-state             6.0.2
sebastian/lines-of-code            2.0.2
sebastian/object-enumerator        5.0.0
sebastian/object-reflector         3.0.0
sebastian/recursion-context        5.0.0
sebastian/type                     4.0.0
sebastian/version                  4.0.1
silber/bouncer                     v1.0.1
spatie/backtrace                   1.5.3
spatie/flare-client-php            1.4.4
spatie/ignition                    1.13.1
spatie/laravel-http-logger         1.11.0
spatie/laravel-ignition            2.5.1
spatie/laravel-package-tools       1.16.4
symfony/console                    v6.4.4
symfony/css-selector               v7.0.3
symfony/deprecation-contracts      v3.4.0
symfony/error-handler              v6.4.4
symfony/event-dispatcher           v7.0.3
symfony/event-dispatcher-contracts v3.4.0
symfony/finder                     v6.4.0
symfony/http-foundation            v6.4.4
symfony/http-kernel                v6.4.5
symfony/mailer                     v6.4.4
symfony/mime                       v6.4.3
symfony/polyfill-ctype             v1.29.0
symfony/polyfill-intl-grapheme     v1.29.0
symfony/polyfill-intl-idn          v1.29.0
symfony/polyfill-intl-normalizer   v1.29.0
symfony/polyfill-mbstring          v1.29.0
symfony/polyfill-php72             v1.29.0
symfony/polyfill-php80             v1.29.0
symfony/polyfill-php83             v1.29.0
symfony/polyfill-uuid              v1.29.0
symfony/process                    v6.4.4
symfony/psr-http-message-bridge    v2.3.1
symfony/routing                    v6.4.5
symfony/service-contracts          v3.4.1
symfony/string                     v7.0.4
symfony/translation                v6.4.4
symfony/translation-contracts      v3.4.1
symfony/uid                        v6.4.3
symfony/var-dumper                 v6.4.4
symfony/yaml                       v6.4.3
theseer/tokenizer                  1.2.3
tijsverkoyen/css-to-inline-styles  v2.2.7
vlucas/phpdotenv                   v5.6.0
voku/portable-ascii                2.0.1
webmozart/assert                   1.11.0

[sebastianbergmann]

Thank you for your report.

Please provide a minimal, self-contained, reproducing test case that shows the problem you are reporting.

Without such a minimal, self-contained, reproducing test case I will not be able to investigate this issue.

[abodnar]

Example unit test:

<?php

namespace Tests\Unit;

use DateTime;
use PHPUnit\Framework\TestCase;

class PrefillDateArrayTest extends TestCase
{
//    private array $defaultArrayItem; <---since this commented out, it will cause the deprecation

    public function setUp(): void
    {
        $this->defaultArrayItem = [];
        parent::setUp();
    }

    public function test(): void
    {
        $from = new DateTime;
        $to = new DateTime('+1 month');

        for ($date = clone $from; $date < $to; $date->modify('+1 day')) {
            $this->defaultArrayItem[$date->format('Y-m-d')] = 1;
        }
        $this->assertIsArray($this->defaultArrayItem);
    }
}

[sebastianbergmann]

I cannot reproduce this:

$ tree -L 2
.
├── composer.json
├── composer.lock
├── phpunit.xml
├── tests
│   └── PrefillDateArrayTest.php
└── vendor
    ├── autoload.php
    ├── bin
    ├── composer
    ├── myclabs
    ├── nikic
    ├── phar-io
    ├── phpunit
    ├── sebastian
    └── theseer

11 directories, 5 files

$ cat composer.json                           
{
    "require-dev": {
        "phpunit/phpunit": "^10.5"
    }
}

$ cat phpunit.xml
<?xml version="1.0" encoding="UTF-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/10.5/phpunit.xsd"
         cacheDirectory=".phpunit.cache"
         colors="true">
    <testsuites>
        <testsuite name="default">
            <directory>tests</directory>
        </testsuite>
    </testsuites>
</phpunit>

$ ./vendor/bin/phpunit
PHPUnit 10.5.16 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.4
Configuration: /home/sb/5793/phpunit.xml

D                                                                   1 / 1 (100%)

Time: 00:00.012, Memory: 8.00 MB

OK, but there were issues!
Tests: 1, Assertions: 1, Deprecations: 1.

$ ./vendor/bin/phpunit --display-deprecations
PHPUnit 10.5.16 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.4
Configuration: /home/sb/5793/phpunit.xml

D                                                                   1 / 1 (100%)

Time: 00:00.008, Memory: 8.00 MB

1 test triggered 1 PHP deprecation:

1) /home/sb/5793/tests/PrefillDateArrayTest.php:14
Creation of dynamic property Tests\Unit\PrefillDateArrayTest::$defaultArrayItem is deprecated

Triggered by:

* Tests\Unit\PrefillDateArrayTest::test
  /home/sb/5793/tests/PrefillDateArrayTest.php:18

OK, but there were issues!
Tests: 1, Assertions: 1, Deprecations: 1.

The deprecation is correctly reported.

[abodnar]

Interesting. I don't know if it matters, but I am doing this with the Laravel sail 8.2 image and have also tried this with the php8.2-cli image. Is there a particular php.ini setting that might dictate if the deprecations are shown?

[sebastianbergmann]

Sure: error_reporting.

[abodnar]

Sigh...of course they have it set to E_ALL & ~E_DEPRECATED & ~E_STRICT in sail. But what's odd is I've added a <ini name="error_reporting" value="32767"/> to my phpunit.xml and also added ini_set('error_reporting', E_ALL); to a setUp method in my base TestCase class and still never displays any summary info about the deprecations.

At this point, it's clearly something with the configuration of Laravel/php/my code that is overriding what I expect to happen. So I appreciate you pointing me at what l needed to look at.

Thanks!