Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI/QA: Switch to GitHub Actions and start using Coveralls #252

Merged
merged 4 commits into from
Jun 1, 2022

Conversation

jrfnl
Copy link
Collaborator

@jrfnl jrfnl commented May 27, 2022

馃憠馃徎 In most repo's which have a similar setup, I normally have "quicktest" and a "test" workflow, where "quicktest" has a smaller matrix and is run on push events and the "test" workflow has the full matrix and is run on pull events (and releases).
This is done based on those repos having a develop and a stable branch. As this repo doesn't have that, I've not modularized the builds that way here.

馃憠馃徎 You can see the results of these workflows on the "Actions tab".

馃憠馃徎 I'd recommend disconnecting the CircleCI integration in the Settings as otherwise the builds will always still show as failed (due to CircleCI not finding a configuration).

馃憠馃徎 The Coveralls configuration may need some tweaking. Suggested settings:
image

CI/QA: Remove CircleCI configuration

... and all work-arounds in place to accommodate. Hope I caught them all.

Composer.json - widen PHPUnit requirements

PHP 5.4 needs PHPUnit 4.x to be able to run the tests, so that needed adding.
Aside from that PHPUnit 9.x has come out quite a while back and has better support for PHP 8.0 and 8.1 than PHPUnit 8.x.

I've set the version constraints for the 4.x and 5.x series to very specific minimum versions for a reason: those are the first versions to have the forward-compatibility layer with the namespaced PHPUnit TestCase, which is used by the tests in this repo.

CI/QA: add GH Actions workflows

This commit adds two workflows which run the same checks as were run in CircleCI + a lot more.

CS and QA workflow: checkcs job

Differences between this workflow and these checks when run via CircleCI:

  • New: This workflow will now also run QA checks on the ruleset.xml file.
    It will run two checks:
    1. Does the ruleset validate against the schema ?
    2. Does the ruleset have a consistent code style, including consistent indentation using four spaces ?
  • Removed: CircleCI ran PHPCS against both PHP 7.3 as well as 7.4. This is redundant as PHPCS should give the same results cross-version. In the new setup, it will only be run once against PHP 7.4.
  • Changed: The PHPCS check will now be run using PHPCS master as an early warning system for upstream bugs.
  • New: By using the cs2pr tool and the xmllint-problem-matcher, violations found will be shown in the GH code views, like the "Files" tab on a PR.

CS and QA workflow: phpstan job

Differences between this workflow and these checks when run via CircleCI:

  • Removed: CircleCI ran PHPStan against both PHP 7.3 as well as 7.4. This is redundant as PHPStan, same like PHPCS, should give the same results cross-version. In the new setup, it will only be run once against PHP 7.4.
  • New: PHPStan natively supports showing violations found in the GH code views, like the "Files" tab on a PR.

Test workflow: test job

  • Changed: CircleCI only ran the tests against PHP 5.4, 5.6, 7.3 and 7.4 and only against the latest version of PHPCS.
    In this new setup, the tests are run against all supported PHP versions + the high/low supported PHPCS, which at this time means: PHPCS 3.5.6 as "low" and PHPCS dev-master as "high".
    This results in a lot more test runs, but this will safeguard the cross-version compatibility of the standard much better as well (and GH Actions is fast, so it shouldn't really impact CI runtime anyway).
  • New: Tests are now also run against PHP 8.0, 8.1 and 8.2, where the run against PHP 8.2 is still allowed to fail for now (continue-on-error).

GH Actions/Test: add code coverage test runs & send to Coveralls

This commits:

  • Splits the test runs into two jobs, where code coverage is run against the high/low PHP/PHPCS combinations and the "normal" test runs remain for all other PHP/PHPCS combination.
  • Includes setting up cache warming for PHPUnit 9.3+ as PHP-Parser can interfere otherwise by backfilling tokens when they are not (yet) available in PHPCS.
  • Adjusts the PHPUnit configuration to allow for measuring code coverage.
  • Adds a new script to the composer.json file for running the tests with code coverage and adjusts the existing test script to ignore code coverage.
  • Adds the badge from Coveralls to the README.

jrfnl added 4 commits May 27, 2022 02:02
... and all work-arounds in place to accommodate. Hope I caught them all.
PHP 5.4 needs PHPUnit 4.x to be able to run the tests, so that needed adding.
Aside from that PHPUnit 9.x has come out quite a while back and has better support for PHP 8.0 and 8.1 than PHPUnit 8.x.

I've set the version constraints for the 4.x and 5.x series to very specific minimum versions for a reason: those are the first versions to have the forward-compatibility layer with the namespaced PHPUnit `TestCase`, which is used by the tests in this repo.
This commit adds two workflows which run the same checks as were run in CircleCI + a lot more.

**CS and QA workflow: `checkcs` job**

Differences between this workflow and these checks when run via CircleCI:
* New: This workflow will now also run QA checks on the `ruleset.xml` file.
    It will run two checks:
    1. Does the ruleset validate against the schema ?
    2. Does the ruleset have a consistent code style, including consistent indentation using four spaces ?
* Removed: CircleCI ran PHPCS against both PHP 7.3 as well as 7.4. This is redundant as PHPCS should give the same results cross-version. In the new setup, it will only be run once against PHP 7.4.
* Changed: The PHPCS check will now be run using PHPCS `master` as an early warning system for upstream bugs.
* New: By using the `cs2pr` tool and the `xmllint-problem-matcher`, violations found will be shown in the GH code views, like the "Files" tab on a PR.

**CS and QA workflow: `phpstan` job**

Differences between this workflow and these checks when run via CircleCI:
* Removed: CircleCI ran PHPStan against both PHP 7.3 as well as 7.4. This is redundant as PHPStan, same like PHPCS, should give the same results cross-version. In the new setup, it will only be run once against PHP 7.4.
* New: PHPStan natively supports showing violations found in the GH code views, like the "Files" tab on a PR.

** Test workflow: `test` job**
* Changed: CircleCI only ran the tests against PHP 5.4, 5.6, 7.3 and 7.4 and only against the latest version of PHPCS.
    In this new setup, the tests are run against all supported PHP versions + the high/low supported PHPCS, which at this time means: PHPCS `3.5.6` as "low" and PHPCS `dev-master` as "high".
    This results in a lot more test runs, but this will safeguard the cross-version compatibility of the standard much better as well (and GH Actions is fast, so it shouldn't really impact CI runtime anyway).
* New: Tests are now also run against PHP 8.0, 8.1 and 8.2, where the run against PHP 8.2 is still allowed to fail for now (`continue-on-error`).
This commits:
* Splits the test runs into two jobs, where code coverage is run against the high/low PHP/PHPCS combinations and the "normal" test runs remain for all other PHP/PHPCS combination.
* Includes setting up cache warming for PHPUnit 9.3+ as PHP-Parser can interfere otherwise by backfilling tokens when they are not (yet) available in PHPCS.
* Adjusts the PHPUnit configuration to allow for measuring code coverage.
* Adds a new script to the `composer.json` file for running the tests with code coverage and adjusts the existing `test` script to ignore code coverage.
* Adds the badge from Coveralls to the README.
@jrfnl jrfnl requested a review from sirbrillig May 27, 2022 01:00
@sirbrillig
Copy link
Owner

Looks good! Thank you!

@sirbrillig sirbrillig merged commit 156490d into 2.x Jun 1, 2022
@sirbrillig sirbrillig deleted the feature/switch-to-ghactions-and-coveralls branch June 1, 2022 23:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants