-
-
Notifications
You must be signed in to change notification settings - Fork 157
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Error "Expected a value identical to 1. Got: 62" when using PHPUnit 10.2 #1869
Comments
I have managed to work around the problem above with The reason is that this is an E2E test suite, where the code under test is running on different processes, and the process running PHPUnit cannot collect code coverage. What I do there is generate code coverage more or less manually with some helpers, and then pass that coverage to infection. |
I downgraded PHPUnit to I get a little bit different error message:
but the stack trace seems to be the same:
|
I think the issues could actually be related. Since I'm generating the code coverage "manually", I might be duplicating tests in the report, because it's not easy to reliably map test methods with source code being run. I will try to solve this and see if there's any difference. |
@maks-rafalko Correct. Created a single <testsuites>
<testsuite name="default">
<directory>tests</directory>
</testsuite>
</testsuites> It's not a solution, its' a workaround. |
In the end, it is actually not related, but it gave me an idea. As I mentioned some comments above, I generate the code coverage myself, because the tests and the source under test are running in different processes, making it impossible for PHPUnit to collect the coverage using the regular approach. The trickier thing for me is determining the code coverage ID consistently. Usually, this ID is a combination of test class + test method name + data provider name. In my case, I couldn't determine the last two reliably, so I went with the class only. That seemed to work in terms of determining percentage of covered code, and to use infection and other tools that require a code coverage report. However, I have realized the junit report which is still generated by PHPUnit, references tests using class + method + data provider, resulting in a mismatch with how they are referenced in the code coverage report. I still don't know if this has anything to do. I'm trying to find a way for my logic to use the three elements as the coverage ID, in order to see if it solves this issue. |
I managed to generate proper coverage IDs, but it didn't solve this. I don't really see a difference in either coverage reports or junit logs from PHPUnit 10.1 to 10.2, so I'm not sure why this fails. I'll continue debugging a bit further later today. |
Ok, it took me some time, but I finally found the difference in the junit.xml file which is causing this. With PHPUnit 10.1, the junit.xml file has an empty <?xml version="1.0" encoding="UTF-8"?>
<testsuites>
<testsuite name="" tests="229" assertions="894" errors="0" failures="0" skipped="0" time="16.916708">
...
</testsuite>
</testsuites> While in PHPUnit 10.2, it has a value containing the path to PHPUnit's config file. <?xml version="1.0" encoding="UTF-8"?>
<testsuites>
- <testsuite name="" tests="229" assertions="894" errors="0" failures="0" skipped="0" time="16.916708">
+ <testsuite name="/home/shlink/phpunit-api.xml" tests="229" assertions="894" errors="0" failures="0" skipped="0" time="16.916708">
...
</testsuite>
</testsuites> If I manually edit that file setting I'm not sure what would be the course of action here though. In other test suites this doesn't seem to be a problem. |
I have tracked down the difference in PHPUnit's code. In v10.1, this call was empty, which results in an empty In v10.2, they pass the config file path: https://github.com/sebastianbergmann/phpunit/blob/10.2.0/src/TextUI/Configuration/Xml/TestSuiteMapper.php#L44 |
Ok, I found the reason for this, and it was on me. I mentioned some comments ago that I generate the code coverage report myself for this particular test suite, and that I had some trouble defining the right coverage IDs. Well, I found a few cases in which the coverage ID was not set at all, generating empty entries. It seems that infection was cross-referencing those with the parent As soon as PHPUnit 10.2 stopped generating that empty name, infection was not able to cross reference those with anything, resulting in this error. It's good, because once I have fixed this, I've found the amount of killed mutants I have is actually higher. This behavior was throwing false negatives. Anyway, I'll close this now 😅 |
When running infection 0.27 together with PHPUnit 10.2 (only in one of my test suites) I get the next error (with very verbose output
-vvv
):I have reproduced this error when I run infection skipping initial tests and instructing it to use already generated coverage and junit log files.
Based on the exception message, it seems PHPUnit 10.2 is generating junit log files with a different structure, but I cannot tell the difference on first sight. I can provide both files if needed.
Everything works properly with PHPUnit 10.1
I'm also trying to to let infection run the tests itself, to make sure it passes in that case. However, I'm finding an unrelated issue which is preventing this, which is that infection seems to be ignoring
testFrameworkOptions
from myinfection-api.json5
file, and is using the wrong phpunit config (it is using default one,phpunit.xml.dist
, which is used for a different test suite).I have tried
--test-framework-options
flag and it seems to be ignored too.I'll try to bypass this somehow and report that issue separately if needed.
The text was updated successfully, but these errors were encountered: