Grouping tests enters both test groups (still executes only one)

[dingo-d]

I ran into a weird problem in my pest implementation for WordPress integration testing, where I'm dependent on two different base test classes for unit and integration tests.

When running the tests, it looks like the pest will enter all the test files regardless of which group is called.

The minimum reproducible example with just pest looks like this:

phpunit.xml:

<?xml version="1.0" encoding="UTF-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:noNamespaceSchemaLocation="./vendor/phpunit/phpunit/phpunit.xsd"
         bootstrap="vendor/autoload.php"
         colors="true"
>
    <testsuites>
        <testsuite name="Unit Test Suite">
            <directory suffix="Test.php">./tests/Unit</directory>
        </testsuite>
        <testsuite name="Integration Test Suite">
            <directory suffix="Test.php">./tests/Integration</directory>
        </testsuite>
    </testsuites>
    <coverage processUncoveredFiles="true">
        <include>
            <directory suffix=".php">./app</directory>
            <directory suffix=".php">./src</directory>
        </include>
    </coverage>
</phpunit>

Pest.php:

<?php

uses()->group('integration')->in('Integration');
uses()->group('unit')->in('Unit');

I have two folders inside the tests folder called Unit and Integration and I've just placed some two dummy tests

IntegrationTest.php:

<?php

var_dump('integration test');

test('integration test', function () {
    expect(true)->toBeTrue();
});

UnitTest.php

<?php

var_dump('unit test');

test('example unit test', function () {
    expect(true)->toBeTrue();
});

When I run the tests using --group option I get the following

vendor/bin/pest --group=integration
/Users/deniszoljom/Projects/Personal/pest-playground/tests/Unit/UnitTest.php:3:
string(9) "unit test"
/Users/deniszoljom/Projects/Personal/pest-playground/tests/Integration/IntegrationTest.php:3:
string(16) "integration test"

   PASS  Tests\Integration\IntegrationTest
  ✓ integration test

  Tests:  1 passed
  Time:   0.01s

And for unit test

vendor/bin/pest --group=unit
/Users/deniszoljom/Projects/Personal/pest-playground/tests/Unit/UnitTest.php:3:
string(9) "unit test"
/Users/deniszoljom/Projects/Personal/pest-playground/tests/Integration/IntegrationTest.php:3:
string(16) "integration test"

   PASS  Tests\Unit\UnitTest
  ✓ example unit test

  Tests:  1 passed
  Time:   0.01s

As you can see, in both cases the var_dump is called from both files.

Why is this a problem?

If I have an inaccessible class that acts as a base test class for my Integration tests (inaccessible meaning if the bootstrap.php file isn't including it), the unit tests will fail with TestCaseClassOrTraitNotFound exception.

You can check my repo to see what I'm talking about.

For unit tests, I am using the Yoast\WPTestUtils\BrainMonkey\TestCase class, which is autoloaded. But for integration tests, I am using the Yoast\WPTestUtils\WPIntegration\TestCase class, which (because of how WordPress is wired) isn't autoloaded via composer, and is only included if the bootstrap.php file includes it (and bootstrap file only includes it for integration tests, as I don't need WordPress for the unit tests of course).

Related issue in my repo is this one.

I'm using pest v1.22.3, tested on MacOS (Laravel Valet) on PHP 7.4.30.

Is this a bug or a feature? And can it be fixed so that group only ever loads the files from a specific group?

Because if not, I'll have to limit my package to only offer the integration tests functionality, and that's a bit limiting :/

EDIT:

I tried just removing the grouping limitation from the bootstrap file, but in this case my unit tests will always use the integration test case. If I manually try to use the unit base test case in the file itself I get

  Test case `Yoast\WPTestUtils\BrainMonkey\TestCase` can not be used. The folder `/Users/deniszoljom/Projects/Personal/wp-pest-playground/tests/Unit/ExampleTest.php` already uses the test case `Yoast\WPTestUtils\BrainMonkey\TestCase`

Which means that integration base test case will always be used, and that's not ok...

[nunomaduro]

Can't reproduce. Sorry! Feel free to pull request something on the v2.x branch.