Skip to content

Conversation

@priethor
Copy link
Contributor

@priethor priethor commented Nov 14, 2025

What

Implement e2e test coverage collection with Codecov integration.

Why

Including only Unit Tests in coverage reports can be misleading, especially when we focus more on e2e testing than unit testing.

How

This involves several steps and changes in the tooling chain:

  • Enabling the coverage collection: Added babel-plugin-istanbul to webpack, activated via COVERAGE_ENABLED=true environment variable and a custom package.json command.
  • Collecting the coverage: Added a base class for all e2e test fixtures to collect coverage from each test. Ideally, this should be directly in wp-scripts.
  • Converting the coverage report: Added .nycrc configuration and npm scripts to generate Codecov-compatible LCOV reports from the Istanbul output.
  • Uploading the coverage report: Updated workflows to upload coverage with appropriate flags (e2e/phpunit) and names.
  • Helping Codecov understand the reports: Updated .codecov.yml settings with separate flags for e2e (assets/src/) and phpunit (includes/) coverage tracking with carryforward enabled. This is necessary because unit and e2e tests run asynchronously and upload coverage independently, and we need Codecov to merge both uploads, "carrying" the other instead of overwriting it.

Testing Instructions

  1. Run npm run test:e2e:coverage locally to generate coverage
  2. Verify ./coverage/lcov.info is generated
  3. Check that coverage HTML report is created in ./coverage/index.html
  4. In CI, verify both e2e and phpunit coverage uploads complete without errors
  5. Check Codecov dashboard shows separate e2e and phpunit flags with combined overall coverage

@codecov
Copy link

codecov bot commented Nov 14, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 7.14%. Comparing base (983ec5c) to head (defde96).
⚠️ Report is 1 commits behind head on trunk.

Additional details and impacted files
@@            Coverage Diff             @@
##             trunk    #244      +/-   ##
==========================================
+ Coverage     3.40%   7.14%   +3.74%     
  Complexity    4423    4423              
==========================================
  Files          261     329      +68     
  Lines        27027   33942    +6915     
==========================================
+ Hits           919    2426    +1507     
- Misses       26108   31516    +5408     
Flag Coverage Δ
e2e 21.79% <ø> (?)
phpunit 3.40% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@priethor priethor marked this pull request as ready for review November 18, 2025 15:31
@github-actions
Copy link

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Core Committers: Use this line as a base for the props when committing in SVN:

Props priethor.

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

@priethor priethor merged commit 6205b2d into trunk Nov 18, 2025
12 checks passed
@priethor priethor deleted the add/coverage-for-e2e-tests branch November 18, 2025 15:31
@priethor priethor changed the title Code coverage: add support to e2e tests Code coverage: add support for e2e tests Nov 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants