PHPCS 4.x | PHP 8.0: backfill the PHP8 namespaced name tokenization #3155
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
As per the proposal in 3041.
This effectively backfills the new PHP 8.0 tokenization of identifier names for all supported PHP versions in PHPCS 4.x.
Note: this backfills takes into account that reserved keywords are now allowed to be used in namespaced names.
For that reason no check is done of the token _type_ of the token between namespace separators, only on the contents.
Includes extensive unit tests to ensure the correct re-tokenization as well as that the rest of the tokenization is not adversely affected by this change.
Includes making sure that the following (re-)tokenizations are still stable:
* Nullable vs ternary.
* Function names are `T_STRING` - but not necessarily so when in a group use statement, i.e. `use Package\{ function Level\functionName };.
* Union types.
…identifier tokens Includes updating the unit tests for the new reality and adding some additional tests.
… identifier tokens Includes adding some additional tests.
…fier tokens Includes adding some additional tests for PHP 8 identifier tokens previously not covered by tests.
…fier tokens Includes adding some additional tests for PHP 8 identifier tokens previously not covered by tests.
…fier tokens The existing unit tests already contain tests covering this change.
Includes adding some additional tests for PHP 8 identifier tokens previously not covered by tests.
... to allow for the different tokenization now expected.
…tifier tokens Includes adding some additional tests for PHP 8 identifier tokens previously not covered by tests.
…P 8 identifier tokens Includes adding some additional tests for PHP 8 identifier tokens previously not covered by tests.
…er tokens The existing unit tests already contain tests covering this change. Includes minor additional fix to make the sniff more tolerant of comments.
…er tokens The existing unit tests already contain a test covering this change. Includes minor additional fix to make the sniff more tolerant of comments.
…entifier tokens The existing unit tests already contain a test covering this change.
…kens The existing unit tests already contain tests covering this change. By extension, this also fixes the `PSR12.Classes.AnonClassDeclaration` and the `Squiz.Classes.ClassDeclaration` sniffs.
… tokens The existing unit tests already contain tests covering this change.
…kens The existing unit tests already contain tests covering this change.
…ifier tokens The existing unit tests already contain tests covering this change.
…fier tokens The existing unit tests already contain tests covering this change. The tests also include a test which would be considered a parse error in PHP 8.0. The current fix will handle this the same in both PHP < 8 as well as PHP 8.0+ and will throw the expected errors even so. The parse error in PHP 8 has been annotated in the test case file.
…ier tokens The existing unit tests already contain tests covering this change.
…r tokens
The existing unit tests already contain tests covering this change.
The sniff has slightly different behaviour PHPCS 4.x vs the previous behaviour in PHPCS 3.x.
Notable differences:
* The tolerance for inline names interlaced with whitespace and comments (parse error in PHP 8) has been removed.
Previously, the sniff would still fix such a name if there was a match. As of PHPCS 4.x it no longer will and won't throw an error for this anymore either.
* The `getDeclarationNameWithNamespace()` method has been removed and the `getNamespaceOfScope()` method has been renamed to `getNamespaceName()`.
Sniffs which extend this sniff and overload(ed) either method, will need to be aware of this.
Includes:
* Fixing an incorrect unit test.
…ifier tokens The existing unit tests already contain tests covering this change. Includes minor additional fix to make the sniff more tolerant of comments.
…kens Includes adding unit tests covering the change.
…kens The existing unit tests already contain a test covering this change.
…dentifier tokens The existing unit tests already contain tests covering this change.
…ier tokens ... to make sure that the spacing for the parentheses of function calls are ignored by this sniff.
... to make sure that function calls using the new tokens are handled correctly by the sniff.
…identifier tokens ... to make sure that the sniff correctly skips to the end of the function call for function calls using the new tokens.
|
Not sure "thanks for the PR" is going to cut it for this one... but thanks! Note that I haven't updated the changelog yet as I don't have time, but have merged this to ensure it doesn't conflict. |
jrfnl
deleted the
feature/phpcs-4.x/3041-use-php8-identifier-tokenization
branch
|
@gsherwood Thanks for merging this this quickly. One more down, only three more to go.... ( |
51 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Now PR #3032 and PR #3152 have been merged, I'm pulling the PR to backfill the PHP 8 tokenization of "namespaced names as single token" in PHPCS 4.x.
Unfortunately I have to pull all the changes in one go as otherwise we wouldn't be able to get a passing build.
@gsherwood It would be greatly appreciated if this PR could get some priority. As it touches so many files, the chances of it getting into a conflict state if left open for a while, are pretty large.
Fixes #3041
Commit Details
PHPCS 4.x / PHP 8.0: namespaced names as single token
As per the proposal in #3041.
This effectively backfills the new PHP 8.0 tokenization of identifier names for all supported PHP versions in PHPCS 4.x.
Note: this backfill takes into account that reserved keywords are now allowed to be used in namespaced names.
For that reason no check is done of the token type of the token between namespace separators, only on the contents.
Includes extensive unit tests to ensure the correct re-tokenization as well as that the rest of the tokenization is not adversely affected by this change.
Includes making sure that the following (re-)tokenizations are still stable:
T_STRING- but not necessarily so when in a group use statement, i.e.use Package\{ function Level\functionName };.Tokenizer/PHP: update the arrow function backfill to allow for PHP 8 identifier tokens
Includes updating the unit tests for the new reality and adding some additional tests.
Tokenizer/PHP: update the short array tokenization to allow for PHP 8 identifier tokens
Includes adding some additional tests.
Tokens::$functionNameTokens: add the new identifier name tokens
File::getMethodParameters(): fix method to work with the PHP 8 identifier tokens
Includes adding some additional tests for PHP 8 identifier tokens previously not covered by tests.
File::getMethodProperties(): fix method to work with the PHP 8 identifier tokens
Includes adding some additional tests for PHP 8 identifier tokens previously not covered by tests.
File::getMemberProperties(): fix method to work with the PHP 8 identifier tokens
The existing unit tests already contain tests covering this change.
File::isReference(): fix method to work with the PHP 8 identifier tokens
Includes adding some additional tests for PHP 8 identifier tokens previously not covered by tests.
File::findEndOfStatement(): update expectations in test
... to allow for the different tokenization now expected.
File::findExtendedClassName(): fix method to work with the PHP 8 identifier tokens
Includes adding some additional tests for PHP 8 identifier tokens previously not covered by tests.
File::findImplementedInterfaceNames(): fix method to work with the PHP 8 identifier tokens
Includes adding some additional tests for PHP 8 identifier tokens previously not covered by tests.
Generic/DuplicateClassName: fix sniff to work with the PHP 8 identifier tokens
The existing unit tests already contain tests covering this change.
Includes minor additional fix to make the sniff more tolerant of comments.
Generic/ForbiddenFunctions: fix sniff to work with the PHP 8 identifier tokens
The existing unit tests already contain a test covering this change.
Includes minor additional fix to make the sniff more tolerant of comments.
Generic/LanguageConstructSpacing: fix sniff to work with the PHP 8 identifier tokens
The existing unit tests already contain a test covering this change.
PSR2/ClassDeclaration: fix sniff to work with the PHP 8 identifier tokens
The existing unit tests already contain tests covering this change.
By extension, this also fixes the
PSR12.Classes.AnonClassDeclarationand theSquiz.Classes.ClassDeclarationsniffs.PSR12/ClassInstantiation: fix sniff to work with the PHP 8 identifier tokens
The existing unit tests already contain tests covering this change.
PSR12/ImportStatement: fix sniff to work with the PHP 8 identifier tokens
The existing unit tests already contain tests covering this change.
PSR12/NullableTypeDeclaration: fix sniff to work with the PHP 8 identifier tokens
The existing unit tests already contain tests covering this change.
PSR12/CompoundNamespaceDepth: fix sniff to work with the PHP 8 identifier tokens
The existing unit tests already contain tests covering this change.
The tests also include a test which would be considered a parse error in PHP 8.0. The current fix will handle this the same in both PHP < 8 as well as PHP 8.0+ and will throw the expected errors even so.
The parse error in PHP 8 has been annotated in the test case file.
Squiz/LowercasePHPFunctions: fix sniff to work with the PHP 8 identifier tokens
The existing unit tests already contain tests covering this change.
Squiz/SelfMemberReference: fix sniff to work with the PHP 8 identifier tokens
The existing unit tests already contain tests covering this change.
The sniff has slightly different behaviour PHPCS 4.x vs the previous behaviour in PHPCS 3.x.
Notable differences:
Previously, the sniff would still fix such a name if there was a match. As of PHPCS 4.x it no longer will and won't throw an error for this anymore either.
getDeclarationNameWithNamespace()method has been removed and thegetNamespaceOfScope()method has been renamed togetNamespaceName().Sniffs which extend this sniff and overload(ed) either method, will need to be aware of this.
Includes:
Squiz/FunctionCommentThrowTag: fix sniff to work with the PHP 8 identifier tokens
The existing unit tests already contain tests covering this change.
Includes minor additional fix to make the sniff more tolerant of comments.
Squiz/VariableComment: fix sniff to work with the PHP 8 identifier tokens
Includes adding unit tests covering the change.
Squiz/OperatorBracket: fix sniff to work with the PHP 8 identifier tokens
The existing unit tests already contain a test covering this change.
Squiz/DisallowMultipleAssignments: fix sniff to work with the PHP 8 identifier tokens
The existing unit tests already contain tests covering this change.
Generic/ArbitraryParenthesesSpacing: add tests with the PHP 8 identifier tokens
... to make sure that the spacing for the parentheses of function calls are ignored by this sniff.
PEAR/FunctionCallSignature: add tests with the PHP 8 identifier tokens
... to make sure that function calls using the new tokens are handled correctly by the sniff.
Squiz/ComparisonOperatorUsageUnitTest: adjust tests to use the PHP 8 identifier tokens
... to make sure that the sniff correctly skips to the end of the function call for function calls using the new tokens.