New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convenience access to contributor "actions" #4167

schlessera opened this Issue Jun 20, 2017 · 2 comments


2 participants

schlessera commented Jun 20, 2017

Across all of our repos, we currently have differing ways of doing something like running the test suite. These come in both .sh and .php files and are spread across utils/, bin/ and ci/ folders.

For the main actions, we should provide a clear and consistent vocabulary, and a convenient (and semi-automated) way of running these actions.


Here's a first try at identifying the main actions, with both the actual actions and there names as a base for dicussion only:


    Before development on the package can start, development dependencies need to be installed, and a test database needs to be created.

  • TEST

    Run the entire test suite against the current state of the package. In case the PREPARE set has not been run yet, let user know and ask if it should be run now.


    Release a new version of the package. This can potentially include building Phar and/or zip files.


The actual implementation is up for discussion as well, with some popular options being:

  • Composer scripts

    composer test

    We already use Composer anyway, and it provides several advantages for PHP code:

    • It abstracts away the location of the binaries, using the local ones, with a fallback to the global ones.
    • Discovery is automated, as the scripts are added to the list of available commands in the Composer help screen.
    • It can directly use PHP code, so using a static method in an autoloaded class is easily done.
    • It abstracts away the difference between code included directly in the repo, and code pulled in as a dependency. So, having the test suite in a separate repo is very easy to do.
  • Shell scripts


    We already have some of these. Flexible and portable, provided we keep within the usual POSIX constraints.

  • PHP scripts

    php scripts/test.php

    We also have some of these. PHP is guaranteed to be included anywhere a WordPress install is running.

  • Makefiles

    make test

    Probably the most popular language-agnostic cross-platform solution. Might not be installed on all systems out-of-the-box.


This comment has been minimized.


schlessera commented Jul 19, 2018

I've implemented this for the refactoring in the following way so far:

  • Everything is based on Composer scripts, for the simple reason that Composer abstracts away the location of binaries, so you can split them up across packages and every individual package can still properly locate them.
  • Tests are now run through composer test for all tests, composer phpunit for PHPUnit tests or composer behat for Behat tests. You can define additional arguments to these testing frameworks by prepending them with a double dash to separate them from Composer arguments. As an example, to run only a single feature file through Behat, you can use composer behat -- features/cli-info.feature. For Behat tests, absence of a features folder skips the tests. For PHPUnit tests, absence of a phpunit.xml[.dist] skips the tests.
  • Linting is run through composer lint. This starts multiple linters in parallel by default to improve performance.
  • Standards & compatibility tests are run through composer phpcs. The tests are skipped if no phpcs.xml[.dist] is found.
  • The database is initialized on first use for the tests through composer prepare-tests.
  • The requirements and the logic for all of these resides in a separate package called wp-cli/wp-cli-tests. More details can be found here:

This comment has been minimized.


GaryJones commented Jul 19, 2018

The tests are skipped if no phpcs.xml[.dist]

PHPCS config files may also start with a period – .phpcs.xml[.dist] – so it should probably check for the presence of them before skipping.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment