Skip to content

fix(workflows): exclude abstract test base classes from Mage-OS suite#373

Merged
mage-os-ci merged 1 commit into
mage-os:mainfrom
rhoerr:fix/full-integration-tests-exclude-abstract-tests
May 17, 2026
Merged

fix(workflows): exclude abstract test base classes from Mage-OS suite#373
mage-os-ci merged 1 commit into
mage-os:mainfrom
rhoerr:fix/full-integration-tests-exclude-abstract-tests

Conversation

@rhoerr
Copy link
Copy Markdown
Contributor

@rhoerr rhoerr commented May 17, 2026

Summary

  • PHPUnit 12 treats discovered Abstract*Test.php files as runner warnings ("Class ... is abstract"). The integration job interprets the resulting non-zero PHPUnit exit code as a failure even when every concrete test in the shard passes.
  • In the most recent full integration run on mage-os/mageos-magento2 (run 25978664094), ~17 of 29 failed jobs are caused entirely by this — for example TestModuleOverrideConfig, GraphQlCache, Customer, Swatches, Test (Integrity), PaypalGraphQl, CatalogUrlRewrite, LayeredNavigation, several Catalog shards, and the Sales/Bundle shards. Test output looks like OK, but there were issues!.
  • This change generates <exclude> entries for any Abstract*Test.php inside the included <directory> shards so only concrete tests load. Single-file shards (*.php matrix entries) are skipped.
  • Drive-by: the trailing echo for the integrationIgnore group had unescaped backticks, causing bash to execute integrationIgnore as a command and log integrationIgnore: command not found in every job. Quoted with single quotes.

Test plan

  • Re-run Integration Tests - Full Test Suite on a branch of mage-os/mageos-magento2 (e.g. release/3.x) and confirm the previously-failing abstract-only jobs now pass.
  • Confirm the Run Integration Tests for Modules step no longer logs integrationIgnore: command not found.
  • Verify the generated phpunit.xml.dist (printed by cat $FILE) includes <exclude> entries for the expected Abstract*Test.php files.

🤖 Generated with Claude Code

PHPUnit 12 treats discovered Abstract*Test.php files as runner warnings,
which the integration job interprets as exit code 1 even when all concrete
tests pass. Generate <exclude> entries for any Abstract*Test.php inside
the included shards. Also fix unescaped backticks in the trailing echo so
bash no longer logs "integrationIgnore: command not found".
@rhoerr rhoerr requested a review from a team as a code owner May 17, 2026 03:29
@mage-os-ci mage-os-ci merged commit eeb9ac2 into mage-os:main May 17, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants