Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI/QA: start recording code coverage #50

Merged
merged 1 commit into from
May 1, 2023

Conversation

jrfnl
Copy link
Collaborator

@jrfnl jrfnl commented May 1, 2023

This commit makes the necessary changes to start recording and uploading code coverage data to Coveralls.

Includes:

  • Updating the GH Actions test worflow to run the high/low PHP version builds with code coverage and upload the results.
  • Updating the PHPUnit config to show an inline code coverage summary and improve the filter configuration.
  • Adding a code coverage badge to the README.

Not included as already in place:

  • Ignoring the potentially generated code coverage files in the build/* directory to prevent it being committed.
  • Adding a script to the composer.json file to run code coverage.

Notes:

  • Coverage will be recorded using Xdebug as the code coverage driver.
  • The php-coveralls/php-coveralls package is used to convert the coverage information from a format as generated by PHPUnit to a format as can be consumed by Coveralls.
    • As this package is only needed for the upload to Coveralls, the package has not been added it to the composer.json file, but will be (global) installed in the GH Actions workflow only.
    • The php-coveralls/php-coveralls package is not 100% compatible with PHP 8.x (yet), so for uploading the coverage generated using PHP 8.x, we switch to PHP 7.4 for uploading the code coverage reports.
  • Coveralls requires a token to identify the repo and prevent unauthorized uploads.
    This token has been added to the repository secrets.
    People with admin access to the GH repo automatically also have access to the admin settings in Coveralls.
    If ever needed, the Coveralls token can be found (and regenerated) in the Coveralls admin for a repo.
    After regeneration, the token as stored in the GH repo Settings -> Secrets and Variables -> Actions -> Repository secrets should be updated.
  • As the workflow sends multiple code coverage reports to Coveralls, we need to tell it to process those reports in parallel and give each report a unique name.
    That's what the COVERALLS_PARALLEL and COVERALLS_FLAG_NAME` settings are about.
  • The coveralls-finish job will signal to Coveralls that all reports have been uploaded.
    This basically gives the "okay" to Coveralls to report on the changes in code coverage via a comment on a GitHub PR as well as via a status check.

The pertinent Coveralls settings used for this repo are:

  • "Only send aggregate Coverage updates to SCM": enabled
  • "Leave comments": enabled
  • "(Comment) Format": compact
  • "Use Status API": enabled
  • "Coverage threshold for failure": 95%
  • "Coverage decrease threshold for failure": 1.0%

This commit make the necessary changes to start recording and uploading code coverage data to Coveralls.

Includes:
* Updating the GH Actions `test` worflow to run the high/low PHP version builds with code coverage and upload the results.
* Updating the PHPUnit config to show an inline code coverage summary and improve the `filter` configuration.
* Adding a code coverage badge to the README.

Not included as already in place:
* Ignoring the potentially generated code coverage files in the `build/*` directory to prevent it being committed.
* Adding a script to the `composer.json` file to run code coverage.

Notes:
* Coverage will be recorded using `Xdebug` as the code coverage driver.
* The `php-coveralls/php-coveralls` package is used to convert the coverage information from a format as generated by PHPUnit to a format as can be consumed by Coveralls.
    - As this package is only needed for the upload to Coveralls, the package has **not** been added it to the `composer.json` file, but will be (global) installed in the GH Actions workflow only.
    - The `php-coveralls/php-coveralls` package is not 100% compatible with PHP 8.x (yet), so for uploading the coverage generated using PHP 8.x, we switch to PHP 7.4 for uploading the code coverage reports.
* Coveralls requires a token to identify the repo and prevent unauthorized uploads.
    This token has been added to the repository secrets.
    People with admin access to the GH repo automatically also have access to the admin settings in Coveralls.
    If ever needed, the Coveralls token can be found (and regenerated) in the Coveralls admin for a repo.
    After regeneration, the token as stored in the GH repo Settings -> Secrets and Variables -> Actions -> Repository secrets should be updated.
* As the workflow sends multiple code coverage reports to Coveralls, we need to tell it to process those reports in parallel and give each report a unique name.
    That's what the `COVERALLS_PARALLEL` and COVERALLS_FLAG_NAME` settings are about.
* The `coveralls-finish` job will signal to Coveralls that all reports have been uploaded.
    This basically gives the "okay" to Coveralls to report on the changes in code coverage via a comment on a GitHub PR as well as via a status check.

The pertinent Coveralls settings used for this repo are:
* "Only send aggregate Coverage updates to SCM": enabled
* "Leave comments": enabled
* "(Comment) Format": compact
* "Use Status API": enabled
* "Coverage threshold for failure": 95%
* "Coverage decrease threshold for failure": 1.0%
@jrfnl jrfnl added this to the 1.x Next Release milestone May 1, 2023
@jrfnl jrfnl merged commit 7da5034 into develop May 1, 2023
@jrfnl jrfnl deleted the JRF/start-recording-code-coverage branch May 1, 2023 19:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant