Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Coverage results in 0% when exported #1003

Closed
acelaya opened this issue Jun 20, 2023 · 1 comment
Closed

Coverage results in 0% when exported #1003

acelaya opened this issue Jun 20, 2023 · 1 comment

Comments

@acelaya
Copy link
Sponsor

acelaya commented Jun 20, 2023

Q A
php-code-coverage version 10.1.2
Driver PCOV
PCOV version 1.0.11
Installation Method Composer
Usage Method Manual

I have a project where I run an E2E test suite with PHPUnit.

In that test suite, the code under test runs in a different process than the tests (a long-running RoadRunner serving a middleware-based app). The tests only perform HTTP requests to a rest API, and assert on the responses.

Because of this I cannot let PHPUnit collect code coverage by itself, as the code under test is never "included" in the process running the tests, and instead, I collect it manually using this package.

This approach has been working fine so far while using openswoole as app server, but now that I'm migrating to RoadRunner, it always results in 0% coverage when exporting to any kind of report (HTML, XML or PHP).

I have debugged the Coverage object just before exporting it, to make sure the code was properly tracked, and everything seems to be there (unless I'm not interpreting properly what I see).

I'm opening this issue because I haven't been able to find any documentation on how to use this library, other than reverse-engineering how PHPUnit uses it itself, and a bit of try and error.

This is a sequence of things that happen when I run my api-tests.sh script:

  1. Start RoadRunner server in background process.
    1. A coverage object is created on bootstrap, and kept in memory.
      use SebastianBergmann\CodeCoverage\CodeCoverage;
      use SebastianBergmann\CodeCoverage\Driver\Selector;
      use SebastianBergmann\CodeCoverage\Filter;
      
      $filter = new Filter();
      $filter->includeDirectory(<src_dir>);
      $coverage = new CodeCoverage((new Selector())->forLineCoverage($filter), $filter);
    2. A PSR-15 middleware is registered that captures code being run during every request:
      function (
          ServerRequestInterface $req,
          RequestHandlerInterface $handler,
      ) use (&$coverage): ResponseInterface {
          $coverage->start($req->getHeaderLine('x-coverage-id'));
      
          try {
              return $handler->handle($req);
          } finally {
              $coverage->stop();
          }
      }
    3. A special route is registered, to "dump" the coverage into a report at the end of the tests.
      use PHPUnit\Runner\Version;
      use SebastianBergmann\CodeCoverage\Report\Html\Facade as Html;
      use SebastianBergmann\CodeCoverage\Report\PHP;
      use SebastianBergmann\CodeCoverage\Report\Xml\Facade as Xml;
      
      // Route is /api-tests/stop-coverage
      
      function () use (&$coverage) {
          $basePath = __DIR__ . '/../../build/coverage-' . $type;
          $covPath = $basePath . '.cov';
      
          var_dump('Covered files: ', $coverage->getReport()->count());
          var_dump('Covered lines: ', $coverage->getReport()->linesOfCode());
          var_dump('Covered classes: ', $coverage->getReport()->numberOfClasses());
          var_dump('Covered functions: ', $coverage->getReport()->functions());
      
          (new Html())->process($coverage, $basePath . '/coverage-html');            
          (new PHP())->process($coverage, $covPath);
          (new Xml(Version::getVersionString()))->process($coverage, $basePath . '/coverage-xml');
      
          return new EmptyResponse();
      }
  2. Run phpunit command: vendor/bin/phpunit -c phpunit-api.xml --log-junit=build/coverage-api/junit.xml
    1. On bootstrap script, register a shutdown function that calls the endpoint that dumps the coverage reports.
      register_shutdown_function(function (): void {
          $httpClient->request(
              'GET',
              'http://localhost:8080/api-tests/stop-coverage',
          );
      });
    2. Every HTTP request these tests do include a X-Coverage-Id header with TestClass::testMethod#dataProviderName.
  3. Stop RoadRunner server.

The var_dumps above do print some numbers, same with openswoole and RoadRunner, but the reports say 0% coverage only when running the tests with RoadRunner.

I may need some help knowing how to debug what I'm doing wrong, or highlighting what I may be doing wrong in the snippets above (as I mentioned, most of it is try and error).

@acelaya
Copy link
Sponsor Author

acelaya commented Jun 20, 2023

Ok, I just realized the problem is that pcov is not capturing any code when executed in the context of RoadRunner.

I did a simple dummy endpoint which basically does this:

$filter = new Filter();
$filter->includeDirectory(<src_dir>);
$driver = new PcovDriver($filter);
$driver->start();

// Use some code from src dir...

var_dump($driver->stop());

It does print some stuff when this endpoint is served with openswoole, but it prints empty arrays when served with RoadRunner.

I will close this for now and investigate further in that direction, or report it to RoadRunner project, as it is perhaps a known issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant