-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Travis: add check to make sure that sniffs are "feature complete" #2364
Travis: add check to make sure that sniffs are "feature complete" #2364
Conversation
9a14d02
to
6366821
Compare
Rebased & updated to account for the changes made in 6ff6b5e |
6366821
to
f199d35
Compare
I've rebased this PR now all the PRs for missing unit test files have been merged. The only sniff still missing unit tests is |
This PR adds a custom script which will check whether each and every sniff is accompanied by a documentation XML file (warning) as well as unit test files (error). Notes: * To only show errors, the script can be run in `quiet` mode by passing the `-q` flag on the command line. * The script has been set up in such a way that it is not only suitable for use by PHPCS itself, but also for use by external standards which use the PHPCS native unit test suite. For external standards to use the tool, run it from the root of the external standards repo like so: `php -f "path/to/PHP_CodeSniffer/scripts/check-sniff-completeness.php"` * The list of sniffs to check completeness for is determined by getting a list of all files in `Sniffs` directories, where the files have a name ending with `Sniff.php` and not prefixed with `Abstract`. * The "has tests" check verifies that a `UnitTest.php` file is present in the standard's `Tests` directory and _at least_ one test case file. * The "has docs" check verifies that a `Standard.xml` file is present in the standard's `Docs` directory. This check only needs to be run on one build as the results won't change between PHP versions. For that reason, I've elected to add it to the fastest PHP version run to balance out the runs some more. As quite a large number of sniffs at this moment do not have documentation files, `quiet` mode is currently enabled for the check in Travis. If/when the missing documentation files have been added, the `-q` flag should be removed and having unit tests as well as documentation should be enforced for all sniffs.
f199d35
to
ba2fc66
Compare
Rebased to fix the merge conflict state. |
@gsherwood Any chance this can be added to the 3.5.0 milestone ? This tool would be very useful for external standards as well. |
Closing in favour of a much improved version of this script now available via https://github.com/PHPCSStandards/PHPCSDevTools If PHPCS still has any interest in using a script like this, I suggest adding the above mentioned repo as a For maintainers of external PHPCS standards: see the readme to learn how to add this check to your CI-toolchain. |
This PR adds a custom script which will check whether each and every sniff is accompanied by a documentation XML file (warning) as well as unit test files (error).
Notes:
quiet
mode by passing the-q
flag on the command line.For external standards to use the tool, run it from the root of the external standards repo like so:
php -f "path/to/PHP_CodeSniffer/scripts/check-sniff-completeness.php"
Sniffs
directories, where the files have a name ending withSniff.php
and not prefixed withAbstract
.UnitTest.php
file is present in the standard'sTests
directory and at least one test case file.Standard.xml
file is present in the standard'sDocs
directory.This check only needs to be run on one build as the results won't change between PHP versions.
For that reason, I've elected to add it to the fastest PHP version run to balance out the runs some more.
As quite a large number of sniffs at this moment do not have documentation files,
quiet
mode is currently enabled for the check in Travis.If/when the missing documentation files have been added, the
-q
flag should be removed and having unit tests as well as documentation should be enforced for all sniffs.Note: the build for this won't pass until all sniffs are accompanied by unit tests.
Currently, the following sniffs do not have unit tests:
Generic.Debug.ClosureLinter
- see PR Generic/ClosureLinter: add missing unit tests #2363Generic.Debug.CSSLint
- see PR Generic/CSSLint: add missing unit tests #2360Generic.Debug.ESLint
- see PR Generic/ESLint: add missing unit tests #2361Generic.Debug.JSHint
- see PR Generic/JSHint: add missing unit tests + allow running without Rhino #2362Generic.PHP.DeprecatedFunctions
- see PR Generic/DeprecatedFunctions: add missing unit tests #2358Generic.VersionControl.SubversionProperties
- see PR Generic/SubversionProperties: add perfunctory unit test files #2359MySource.Channels.IncludeOwnSystem
- to be addressed, possibly in the same way as Generic/SubversionProperties: add perfunctory unit test files #2359.As it isn't completely clear to me without access to the proprietary system for which this sniff was written, what and how it should be tested, this is for someone else to sort out.
Related: #2345 (comment)