-
-
Notifications
You must be signed in to change notification settings - Fork 352
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
junit
report is broken
#48
Comments
That sounds normal to me. Behind the scenes pest creates real classes to make the integration seamless with PHPUnit. Not sure if we plan to fix this. |
That's perfectly fine, but let me disagree with you that the current So the purpose of
However, we see that the
Two issues here
And now, let me describe the importance of the Imagine, you have 1000 tests, and only 2 of them covers line 33 that is currently being mutated by Infection. Infection does its best to run only those tests that covers the mutated line (line 33 in our example). All other 998 tests are not executed. And it does that thanks to the information fetched from Do you have any ideas how we can improve that? |
Got it. Without junit report, no infection adapter right? |
Well, we can do it, but it will be barely useful. The worst thing is the time. Another example:
Let me know if I can help with anything here. |
I will take a better look at this tomorrow. |
Just pushed a very native solution for this: https://github.com/pestphp/pest/tree/feat/poc-junit. Can you try it out and tell me if something is missing? |
It's much better! But still not enough. Currently, it generates such <?xml version="1.0" encoding="UTF-8"?>
<testsuites>
<testsuite name="" tests="1" assertions="1" errors="0" warnings="0" failures="0" skipped="0" time="0.022643">
<testsuite name="P\Tests\SourceFileTest" file="/pest/src/Factories/TestCaseFactory.php(187) : eval()'d code" tests="1" assertions="1" errors="0" warnings="0" failures="0" skipped="0" time="0.022643">
<testcase name="it can add two integers" class="Tests\SourceFileTest" classname="Tests.SourceFileTest" file="/pest-example/tests/SourceFileTest.php" assertions="1" time="0.022643"/>
</testsuite>
</testsuite>
</testsuites> where but node Can we somehow have this: <?xml version="1.0" encoding="UTF-8"?>
<testsuites>
<testsuite name="" tests="1" assertions="1" errors="0" warnings="0" failures="0" skipped="0" time="0.022643">
- <testsuite name="P\Tests\SourceFileTest" file="/pest/src/Factories/TestCaseFactory.php(187) : eval()'d code" tests="1" assertions="1" errors="0" warnings="0" failures="0" skipped="0" time="0.022643">
+ <testsuite name="P\Tests\SourceFileTest" file="/pest-example/tests/SourceFileTest.php" tests="1" assertions="1" errors="0" warnings="0" failures="0" skipped="0" time="0.022643">
<testcase name="it can add two integers" class="Tests\SourceFileTest" classname="Tests.SourceFileTest" file="/pest-example/tests/SourceFileTest.php" assertions="1" time="0.022643"/>
</testsuite>
</testsuite>
</testsuites> or at least this <?xml version="1.0" encoding="UTF-8"?>
<testsuites>
<testsuite name="" tests="1" assertions="1" errors="0" warnings="0" failures="0" skipped="0" time="0.022643">
<testsuite name="P\Tests\SourceFileTest" file="/pest/src/Factories/TestCaseFactory.php(187) : eval()'d code" tests="1" assertions="1" errors="0" warnings="0" failures="0" skipped="0" time="0.022643">
- <testcase name="it can add two integers" class="Tests\SourceFileTest" classname="Tests.SourceFileTest" file="/pest-example/tests/SourceFileTest.php" assertions="1" time="0.022643"/>
+ <testcase name="it can add two integers" class="P\Tests\SourceFileTest" classname="Tests.SourceFileTest" file="/pest-example/tests/SourceFileTest.php" assertions="1" time="0.022643"/>
</testsuite>
</testsuite>
</testsuites> In other words, we need to have a node where we both have Explanation:
|
@nunomaduro with the following 2 changes in your branch, I was able to generate a correct diff --git a/native/phpunit/phpunit/src/Util/Log/JUnit.php b/native/phpunit/phpunit/src/Util/Log/JUnit.php
index 3a21ead..87afb77 100644
--- a/native/phpunit/phpunit/src/Util/Log/JUnit.php
+++ b/native/phpunit/phpunit/src/Util/Log/JUnit.php
@@ -198,8 +198,7 @@ final class JUnit extends Printer implements TestListener
if ($suite instanceof \Pest\Contracts\Test) {
$testSuite->setAttribute('file', $suite->getFilename());
} else {
- $class = new \ReflectionClass($suite->getName());
- $testSuite->setAttribute('file', $class->getFileName());
+ $testSuite->setAttribute('file', $suite->tests()[0]->getFilename());
}
} catch (\ReflectionException $e) {
}
@@ -303,7 +302,7 @@ final class JUnit extends Printer implements TestListener
// @codeCoverageIgnoreEnd
if ($test instanceof \Pest\Contracts\Test) {
- $testCase->setAttribute('class', $test->getPrintableTestCaseName());
+ $testCase->setAttribute('class', $class->getName());
$testCase->setAttribute('classname', \str_replace('\\', '.', $test->getPrintableTestCaseName()));
$testCase->setAttribute('file', $test->getFilename());
} else { I've created 2 pest files (tests) with 2 cases ( Resulting <?xml version="1.0" encoding="UTF-8"?>
<testsuites>
<testsuite name="" tests="4" assertions="4" errors="0" warnings="0" failures="0" skipped="0" time="0.028040">
<testsuite name="P\Tests\FirstSourceFileTest" file="/pest-example/tests/FirstSourceFileTest.php" tests="2" assertions="2" errors="0" warnings="0" failures="0" skipped="0" time="0.021014">
<testcase name="it First file - first case" class="P\Tests\FirstSourceFileTest" classname="Tests.FirstSourceFileTest" file="/pest-example/tests/FirstSourceFileTest.php" assertions="1" time="0.019044"/>
<testcase name="it First file - second case" class="P\Tests\FirstSourceFileTest" classname="Tests.FirstSourceFileTest" file="/pest-example/tests/FirstSourceFileTest.php" assertions="1" time="0.001970"/>
</testsuite>
<testsuite name="P\Tests\SecondSourceFileTest" file="/pest-example/tests/SecondSourceFileTest.php" tests="2" assertions="2" errors="0" warnings="0" failures="0" skipped="0" time="0.007026">
<testcase name="it Second file - first case" class="P\Tests\SecondSourceFileTest" classname="Tests.SecondSourceFileTest" file="/pest-example/tests/SecondSourceFileTest.php" assertions="1" time="0.003728"/>
<testcase name="it Second file - second case" class="P\Tests\SecondSourceFileTest" classname="Tests.SecondSourceFileTest" file="/pest-example/tests/SecondSourceFileTest.php" assertions="1" time="0.003297"/>
</testsuite>
</testsuite>
</testsuites> For sure, this is not a ready solution, but probably will speed up your investigation on this topic. What do you think we need to do else to move it forward? |
To be honest I am without time for this at the moment. But it's cool that we already know how to fix this problem in pest. |
This issue is actually related to the issues we had with TeamCity printer. |
I’m closing this issue because it has been inactive for a few months. |
@nunomaduro @maks-rafalko @olivernybroe Is it okay that Infection shows the same in its report when using Pest v2? It seems the issue appeared again: |
@michael-rubel Pest v2 does not have junit support. |
@olivernybroe 😄 For some time after upgrading to v2 I was sure my code is "ideal" because Infection was reporting 100% when I was at 100% coverage. Incredible fail. |
It's not entirely clear to me whether this is still an open issue. @olivernybroe mentioned that Pest v2 does not have junit support; is this still the case? I see that I'm running into this today because this causes Gitlab coverage reports to break because the correct filename can't be determined. If this is still an issue, are there any workarounds? |
@arondeparon issue was resolved back in pest V1, hence why it was closed. @nunomaduro chose to not implement it in pest V2, but I am pretty sure that he is okay with someone adding support. The best workaround would be to use team city output as that is supported. There might exist converters between those formats. Better workaround is to submit a PR 😉 |
How to reproduce:
Source class:
test file:
Command line:
vendor/bin/pest --coverage-xml=pest-coverage-xml --log-junit="pest-coverage-xml/junit.xml"
generated files:
And the incorrect
junit.xml
report looks like:What's wrong here? look at the following line:
and in particular look at the
file
attribute:It's not a valid test file.
❗❗
Expected:
/pest-example/tests/SourceFileTest.php
Actual:
/pest-example/vendor/pestphp/pest/src/Factories/TestCaseFactory.php(185) : eval()'d code
If I run the same test but written in PHPUnit style, I get the correct
junit
report:Related to and blocks #4
The text was updated successfully, but these errors were encountered: