Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JUnit formatter incorrectly reports the total number of tests run #6772

AlexHowansky opened this issue Mar 9, 2022 · 5 comments · Fixed by phpstan/phpstan-src#1722


Copy link

Bug report

Using v1.4.8

When logging via the JUnit formatter, the tests attribute in the testsuite element gets set to the number of failures. It should get set to the total number of tests performed.

Code snippet that reproduces the problem

vendor/bin/phpstan --error-format=junit

This is the log for a run with 50 tests, 1 of which fails:

<?xml version="1.0" encoding="UTF-8"?>
<testsuite failures="1" name="phpstan" tests="1" xmlns:xsi="...">

Expected output

Via this JUnit doc page:

Child elements: <testsuite>
  id: The ID of the scan.
  name: The label of the scan.
  tests: The total number of rules that were applied.
  failures: The total number of rule violations.
  time The time that was required to process all the rules.

Note the number of tests is known to be 50 because of the progress meter:

50/50 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓] 100%

The expected output is thus:

<?xml version="1.0" encoding="UTF-8"?>
<testsuite failures="1" name="phpstan" tests="50" xmlns:xsi="...">

The main side effect of this issue is that when all tests pass, the log contains tests="0" and a CI/CD pipeline scanning that log reports that nothing was done. For example, here is BitBucket showing the "Static Analysis" step which should have "50 tests passed":


Did PHPStan help you today? Did it make you happy in any way?

Copy link

mergeable bot commented Mar 9, 2022

This bug report is missing a link to reproduction on

It will most likely be closed after manual review.

Copy link

What number would you expect there? I can't think of anything that would make sense.

Copy link

The number that the progress bar displays seemed the obvious choice to me. What does that normally represent, the number of files processed?

Copy link

mglaman/phpstan-junit and Psalm both report the suite as having 1 test if there are no failures/errors (links point to their respective code snippets).

Below is a screen capture demonstrating Psalm's "1 tests passed".

Screen capture of Psalm JUnit test report in Bitbucket Pipelines interface

Copy link

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 16, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
None yet

Successfully merging a pull request may close this issue.

3 participants