Skip to content

Commit

Permalink
CI/QA: start recording code coverage
Browse files Browse the repository at this point in the history
This commit make the necessary changes to start recording and uploading code coverage data to Coveralls.

Includes:
* Updating the GH Actions `test` worflow to run the high/low PHP version builds with code coverage and upload the results.
* Updating the PHPUnit config to show an inline code coverage summary and improve the `filter` configuration.
* Adding a code coverage badge to the README.

Not included as already in place:
* Ignoring the potentially generated code coverage files in the `build/*` directory to prevent it being committed.
* Adding a script to the `composer.json` file to run code coverage.

Notes:
* Coverage will be recorded using `Xdebug` as the code coverage driver.
* The `php-coveralls/php-coveralls` package is used to convert the coverage information from a format as generated by PHPUnit to a format as can be consumed by Coveralls.
    - As this package is only needed for the upload to Coveralls, the package has **not** been added it to the `composer.json` file, but will be (global) installed in the GH Actions workflow only.
    - The `php-coveralls/php-coveralls` package is not 100% compatible with PHP 8.x (yet), so for uploading the coverage generated using PHP 8.x, we switch to PHP 7.4 for uploading the code coverage reports.
* Coveralls requires a token to identify the repo and prevent unauthorized uploads.
    This token has been added to the repository secrets.
    People with admin access to the GH repo automatically also have access to the admin settings in Coveralls.
    If ever needed, the Coveralls token can be found (and regenerated) in the Coveralls admin for a repo.
    After regeneration, the token as stored in the GH repo Settings -> Secrets and Variables -> Actions -> Repository secrets should be updated.
* As the workflow sends multiple code coverage reports to Coveralls, we need to tell it to process those reports in parallel and give each report a unique name.
    That's what the `COVERALLS_PARALLEL` and COVERALLS_FLAG_NAME` settings are about.
* The `coveralls-finish` job will signal to Coveralls that all reports have been uploaded.
    This basically gives the "okay" to Coveralls to report on the changes in code coverage via a comment on a GitHub PR as well as via a status check.

The pertinent Coveralls settings used for this repo are:
* "Only send aggregate Coverage updates to SCM": enabled
* "Leave comments": enabled
* "(Comment) Format": compact
* "Use Status API": enabled
* "Coverage threshold for failure": 95%
* "Coverage decrease threshold for failure": 1.0%
  • Loading branch information
jrfnl committed May 1, 2023
1 parent b194794 commit b4d180a
Show file tree
Hide file tree
Showing 3 changed files with 52 additions and 5 deletions.
49 changes: 47 additions & 2 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,15 @@ jobs:

strategy:
matrix:
php: ['5.6', '7.0', '7.1', '7.2', '7.3', '7.4', '8.0', '8.1', '8.2', '8.3']
php: ['7.0', '7.1', '7.2', '7.3', '7.4', '8.0', '8.1', '8.3']
coverage: [false]

# Run code coverage only on high/low PHP.
include:
- php: 5.6
coverage: true
- php: 8.2
coverage: true

continue-on-error: ${{ matrix.php == '8.3' }}

Expand All @@ -29,7 +37,7 @@ jobs:
with:
php-version: ${{ matrix.php }}
ini-values: error_reporting=E_ALL, display_errors=On
coverage: none
coverage: ${{ matrix.coverage == true && 'xdebug' || 'none' }}

# Install dependencies and handle caching in one go.
# @link https://github.com/marketplace/actions/install-composer-dependencies
Expand All @@ -52,4 +60,41 @@ jobs:
run: composer lint

- name: Run the unit tests
if: ${{ matrix.coverage == false }}
run: composer test

- name: Run the unit tests with code coverage
if: ${{ matrix.coverage == true }}
run: composer coverage

# PHP Coveralls doesn't fully support PHP 8.x yet, so switch the PHP version.
- name: Switch to PHP 7.4
if: ${{ success() && matrix.coverage == true && startsWith( matrix.php, '8' ) }}
uses: shivammathur/setup-php@v2
with:
php-version: 7.4
coverage: none

# Global install is used to prevent a conflict with the local composer.lock in PHP 8.0+.
- name: Install Coveralls
if: ${{ success() && matrix.coverage == true }}
run: composer global require php-coveralls/php-coveralls:"^2.5.3" --no-interaction

- name: Upload coverage results to Coveralls
if: ${{ success() && matrix.coverage == true }}
env:
COVERALLS_REPO_TOKEN: ${{ secrets.COVERALLS_TOKEN }}
COVERALLS_PARALLEL: true
COVERALLS_FLAG_NAME: php-${{ matrix.php }}
run: php-coveralls -v -x build/logs/clover.xml

coveralls-finish:
needs: test
runs-on: ubuntu-latest

steps:
- name: Coveralls Finished
uses: coverallsapp/github-action@v2
with:
github-token: ${{ secrets.COVERALLS_TOKEN }}
parallel-finished: true
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ WP Test Utils
[![Version](https://poser.pugx.org/yoast/wp-test-utils/version)](https://packagist.org/packages/yoast/wp-test-utils)
[![CS Build Status](https://github.com/Yoast/wp-test-utils/actions/workflows/cs.yml/badge.svg)](https://github.com/Yoast/wp-test-utils/actions/workflows/cs.yml)
[![Test Build Status](https://github.com/Yoast/wp-test-utils/actions/workflows/test.yml/badge.svg)](https://github.com/Yoast/wp-test-utils/actions/workflows/test.yml)
[![Coverage Status](https://coveralls.io/repos/github/Yoast/wp-test-utils/badge.svg?branch=develop)](https://coveralls.io/github/Yoast/wp-test-utils?branch=develop)

[![Minimum PHP Version](https://img.shields.io/packagist/php-v/yoast/wp-test-utils.svg?maxAge=3600)](https://packagist.org/packages/yoast/wp-test-utils)
[![License: BSD3](https://poser.pugx.org/yoast/wp-test-utils/license)](https://github.com/Yoast/wp-test-utils/blob/main/LICENSE)

Expand Down
6 changes: 3 additions & 3 deletions phpunit.xml.dist
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,14 @@
<directory suffix=".php">./src/</directory>
<exclude>
<file>./src/BrainMonkey/bootstrap.php</file>
<!-- The functionality within these files cannot be tested without installing WP. -->
<file>./src/WPIntegration/bootstrap-functions.php</file>
<file>./src/WPIntegration/Autoload.php</file>
<!-- The functionality for WP Integration tests cannot be tested without installing WP. -->
<directory suffix=".php">./src/WPIntegration/</directory>
</exclude>
</whitelist>
</filter>

<logging>
<log type="coverage-text" target="php://stdout" showOnlySummary="true"/>
<log type="coverage-clover" target="build/logs/clover.xml"/>
</logging>

Expand Down

0 comments on commit b4d180a

Please sign in to comment.