Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update code to comply with WP Coding Standards (*fourth* time's the charm!) #50

Merged
merged 70 commits into from
Feb 28, 2021

Conversation

Rumperuu
Copy link
Collaborator

Description of Changes

Updates the codebase to ensure compliance with the WP Coding Standard.

Also Composerises the project to allow for the installation of a pre-commit PHP_CodeSniffer hook (as well as other hooks later on).

History

This is a copy of #38 after it was un-merged in #49.

Additional Notes

This branch's commit history will be squashed on merge, so do not make any commits to this branch that you're going to want to remain in the project history. Keep it scoped to WP Coding Standards compliance only.

Actions

Replace #38

Close #8
Close #10
Close #11
Close #46

pewgeuges and others added 30 commits February 20, 2021 18:18
…gfix.

= 2.5.6 =
- Bugfix: Reference container: optional alternative expanding and collapsing without jQuery for use with hard links, thanks to @hopper87it @pkverma99 issue reports.
- Bugfix: Alternative tooltips: shrink width to short content.
- Update: Documentation: slightly revise or update the plugin’s welcome page on WordPress.org.

git-svn-id: https://plugins.svn.wordpress.org/footnotes/trunk@2478170 b8457f37-d9ea-0310-8a92-e5e31aec5664
This is a squashed copy of `wp-coding-standard--rebase` to try and fix the line-ending
diff issue we're having.
Not doing so will result in a `WordPress.WP.EnqueuedResourceParameters.NotInFooter` warning from PHPCS.
Merge deleted docblock attributions in remaining docblock.
Update: Libraries: Load jQuery UI from WordPress, thanks to @check2020de issue report.
Under 2.0.4.
Name variables in an intuitive manner according to coding conventions.
Fix header docblock as discussed in #38 (comment)
Fix 2 function docblock headings by dispatching them from the originally single metabox function docblock.
Also add version tags to the 2 metaboxes split off @SInCE 2.2.0.
* Displays all options for the footnotes start and end tag short codes. (Initially called “Footnotes styling”.)
* Displays all options for the footnotes numbering. (Newly individualized.)
* Displays all options for the scrolling behavior. (Totally new as scroll offset and duration were initially hard-coded in the template.)
(This is part of WP coding standard compliance.)
Titlecase “Translators:”.
Add semicolons.
Headings and version for split-off metaboxes docblocks.
Overwrites the todo items.
Thanks for starting the docblocks required by WP coding standards.
The historic repo where @felipelavinz contributed has recently been moved.
As a consequence the commit’s URL needs to be updated.
But the original URL is maintained below under the commit date.
@Rumperuu Rumperuu removed the security Anything with security implications label Feb 27, 2021
@Rumperuu
Copy link
Collaborator Author

Rumperuu commented Feb 27, 2021

AFAICS, you fixed the header output, where minified or halfways minified CSS makes actually some sense except for support, which shouldn’t be a reason to not minify given we can reformat it if needed.
The bug I referred to was happening in the dashboard, where Custom CSS is minified with spaces on save, something inacceptable for users.

My mistake, I misunderstood where you were having the issue.

Moreover, HTML tags are deleted (stripped off) from the labels, where users may add them. E.g. https://wordpress.org/support/topic/how-do-i-eliminate-the-horizontal-line-beneath-the-reference-container-heading/ shows how a user added markup to the label, for a suboptimal result that was subsequently fixed and triggererd the addition of 2 new settings, one of which to select the element natively. But stripping off these tags is inacceptable for users. Also witnessed in referrers (string before/after). Escaping all this is detrimental and it’s also pointless, as it is not clear to what avail this overkill security scheme is enforced in the dashboard of a plugin in an environment where other plugins can help add heaps of internal scripts. No need to hijack any Footnotes setting to do that.

The real use of escapement is to address security issues about visitor input, where everyone out there can try to inject malicious code. That however is not a concern in the Footnotes plugin.

On top of all that, even the optional Footnotes ad string (e.g. I love Footnotes) to be displayed in the footer of the public pages is now escaped.

Agreed, the use of output escaping is not necessary within the context of this Plugin. For now, I've removed all of the output escape and input sanitisation functions in 25c3f2f (you can check by running grep -r 'esc_\|wp_kses_\|sanitize_\|wp_filter_' . --exclude-dir={.git/,.phpdoc/,vendor/,docs/,languages/}) and replaced them with file-level PHPCS suppressions for those rules. I'll have a proper look into whether the same is true of input sanitisation (i.e., for $_REQUEST and $_POST values) when I get around to tackling #52, but for now the errors are suppressed so that's considered fixed within the scope of this PR.

I can’t seem to understand why we should turn the validator off at every non-escaped output variable instead of editing the tools’ global settings for this codebase.

The use of file-level suppression comments solves this.

This branch has issues that make it a threat to the plugin’s stability and usability. It disrupts the plugin’s dashboard UX and capabilities, see #50 (comment)

Fixed in 25c3f2f

Also, this branch goes beyond its scope by making these disruptive changes to the code.

Incorrect. The scope of this PR is to add pre-commit linting to enforce compliance with the WP Coding Standards. The linter raises errors for unescaped output, and so solving that issue (whether by escaping output or suppressing the error) is within this scope.

Can the excess be safely reverted, or would it be more efficient to restart from zero (2.5.6, or better, now 2.5.7, or oncoming 2.6.0 if AMP compat can be provided and released within a useful delay)?

As @markcheret has said elsewhere, the current priority is getting the codebase up-to-snuff (provided there are no critical/security fixes to implement).

I think best would be to clearly design the scope and the steps, e.g.:

As stated above, the scope of this branch is clear: implement pre-commit PHPCS validation to enforce compliance with the WP Coding Standards, and ensure that all code passes. This necessarily involves a huge number of minor changes (e.g., double quotes to single quotes), but it also requires a few manual changes that are more impactful and have the potential to introduce problems (e.g., the output escaping). These manual changes are where issues may arise, but they make up a minority of the overall changeset.

Sadly that would invalidate much of your hard work. But Footnotes’ users are top priority and must not become the victims of project-internal issues.

Again as per @markcheret—excluding any critical/security issues that may arise—merging this PR is currently the priority. Once we've done so, we can return to regular bugfixing and adding features on the standardised codebase (rather than maintaining two variants—it required a non-trivial amount of effort to backport the 2.5.7 bugfix in #51 and that only altered 4 or 5 lines).

As the standardisation of the codebase will not merit a new version of the Plugin, we will still have plenty of time still to catch any other issues that may slip through during subsequent normal development, though between your thorough code review and the reversion of output escaping in 25c3f2f (which was one of the most widespread impactful manual changes made) I think that is unlikely.

In a worst-case scenario, even if something really obscure breaks and we don't notice until we get a bug report in five months time, then that's something we can fix then and there easily enough. Only 50% of users are on the latest version, after all.

@pewgeuges
Copy link
Contributor

pewgeuges commented Feb 27, 2021

AFAICS, you fixed the header output, where minified or halfways minified CSS makes actually some sense except for support, which shouldn’t be a reason to not minify given we can reformat it if needed.
The bug I referred to was happening in the dashboard, where Custom CSS is minified with spaces on save, something inacceptable for users.

My mistake, I misunderstood where you were having the issue.

No problem.

Moreover, HTML tags are deleted (stripped off) from the labels, […]

Agreed, the use of output escaping is not necessary within the context of this Plugin. For now, I've removed all of the output escape and input sanitisation functions in 25c3f2f (you can check by running grep -r 'esc_\|wp_kses_\|sanitize_\|wp_filter_' . --exclude-dir={.git/,.phpdoc/,vendor/,docs/,languages/}) and replaced them with file-level PHPCS suppressions for those rules.

Thank you for fixing the escapement problem.

I'll have a proper look into whether the same is true of input sanitisation (i.e., for $_REQUEST and $_POST values) when I get around to tackling #52, but for now the errors are suppressed so that's considered fixed within the scope of this PR.

$_POST is used for the dashboard, so you won’t need to change anything, because the input is expected to stay unaltered as it is.

I can’t seem to understand why we should turn the validator off at every non-escaped output variable instead of editing the tools’ global settings for this codebase.

The use of file-level suppression comments solves this.

File level configuration seems a good compromise between project-wide and local.

This branch has issues that make it a threat to the plugin’s stability and usability. It disrupts the plugin’s dashboard UX and capabilities, see #50 (comment)

Fixed in 25c3f2f

Thanks!

Also, this branch goes beyond its scope by making these disruptive changes to the code.

Incorrect. The scope of this PR is to add pre-commit linting to enforce compliance with the WP Coding Standards. The linter raises errors for unescaped output, and so solving that issue (whether by escaping output or suppressing the error) is within this scope.

WordPress Coding Standards does not seem to recommend Lint. The compiler catches bugs, while a linter seems to enforce general or common-sense rules that may, as you state, not apply to a given codebase.

Can the excess be safely reverted, or would it be more efficient to restart from zero (2.5.6, or better, now 2.5.7, or oncoming 2.6.0 if AMP compat can be provided and released within a useful delay)?

As @markcheret has said elsewhere, the current priority is getting the codebase up-to-snuff (provided there are no critical/security fixes to implement).

I tried to carefully evaluate @markcheret’s directives and didn’t suspect that I got him wrong, as he both mentioned “compliance” AND “Let's also continue what has been going on bugfixes and enhancements can be done alongside if the codebase permits.” And: “So, if there are improvements to publish, by all means, go for it and publish a tested version of new bugfixes or features.” The “alongside” is what allowed to release bugfix 2.5.7 in real time, which is also what I anticipated the possible need of.

I think best would be to clearly design the scope and the steps, e.g.:

As stated above, the scope of this branch is clear: implement pre-commit PHPCS validation to enforce compliance with the WP Coding Standards, and ensure that all code passes. This necessarily involves a huge number of minor changes (e.g., double quotes to single quotes),

I’m fine with that, no problem for me. It’s even easier to type.

but it also requires a few manual changes that are more impactful and have the potential to introduce problems (e.g., the output escaping).

I’m sorry that the linter or whatever tool got you to go to that trouble.

These manual changes are where issues may arise, but they make up a minority of the overall changeset.

I’m glad if that’s all.

Sadly that would invalidate much of your hard work. But Footnotes’ users are top priority and must not become the victims of project-internal issues.

Again as per @markcheret—excluding any critical/security issues that may arise—merging this PR is currently the priority. Once we've done so, we can return to regular bugfixing and adding features on the standardised codebase (rather than maintaining two variants—it required a non-trivial amount of effort to backport the 2.5.7 bugfix in #51

Thank you very much for integrating the 2.5.7 changes!

and that only altered 4 or 5 lines).

Indeed 2.5.7 was basically 1 line of code and 1 new docblock above it.

As the standardisation of the codebase will not merit a new version of the Plugin, we will still have plenty of time still to catch any other issues that may slip through during subsequent normal development, though between your thorough code review and the reversion of output escaping in 25c3f2f (which was one of the most widespread impactful manual changes made) I think that is unlikely.

Seems unlikely, I think so too. Will just review the code again, hoping to catch any remainder.

In a worst-case scenario, even if something really obscure breaks and we don't notice until we get a bug report in five months time, then that's something we can fix then and there easily enough. Only 50% of users are on the latest version, after all.

Sigh.

Development Workflow automation moved this from Review in progress to Reviewer approved Feb 28, 2021
Copy link
Contributor

@pewgeuges pewgeuges left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’m unable to review the codebase again, and it’s superfluous since @Rumperuu assessed all problems as solved.

Moreover the plugin escapes quotation marks in user input! It’s unusable for custom CSS involving HTML argument values. The plugin is still so buggy it would be shameless to submit it to AMP-WP as-is even after adding AMP compat.

RTL compat bugfix v2.5.8 needs to be fast-tracked instead, see the latest support issue.

@pewgeuges pewgeuges mentioned this pull request Feb 28, 2021
@markcheret
Copy link
Owner

@pewgeuges @Rumperuu is the escaping of quotation marks in the user input a new behaviour introduced by the refactoring here or has it historically been implemented?

@Rumperuu
Copy link
Collaborator Author

Rumperuu commented Feb 28, 2021

RTL compat bugfix v2.5.8 needs to be fast-tracked instead, see the latest support issue.

That's fine, 2.5.8 shouldn't be any bother to backport.

@pewgeuges @Rumperuu is the escaping of quotation marks in the user input a new behaviour introduced by the refactoring here or has it historically been implemented?

I'm not able to reproduce this on either main or wp-coding-standard--fresh. See here for my custom CSS (with quotation marks) and here the output. @pewgeuges have you definitely pulled the branch since 25c3f2f?

Update: HTML elements within CSS are escaped; see here for custom CSS and here for output. This is also the case on main. I'm not sure if this is what @pewgeuges was referring to, however if it is then this is expected behaviour per the CSS spec.

pewgeuges and others added 2 commits February 28, 2021 14:24
Will be released in response to Forum and as agreed.
= 2.5.8 =
- Bugfix: Layout: support right-to-left writing direction by replacing remaining CSS values 'left' with 'start', thanks to @arahmanshaalan bug report.
- Bugfix: Layout: support right-to-left writing direction by enabling mirrored paddings on HTML dir=rtl pages, thanks to @arahmanshaalan bug report.
@Rumperuu Rumperuu mentioned this pull request Feb 28, 2021
@pewgeuges
Copy link
Contributor

pewgeuges commented Feb 28, 2021

RTL compat bugfix v2.5.8 needs to be fast-tracked instead, see the latest support issue.

That's fine, 2.5.8 shouldn't be any bother to backport.

I did backport it, even prior to releasing it, but the line end issue has come back. Why is this branch using other line ends, and what can I do to address the problem?

Given this branch does not edit the style sheets, I pasted the modified ones over the existing ones. E.g. 12 minified style sheets were changed.

As of the readme.txt, I synced it alongside so it was handled the same way. Only footnotes.php was edited in the Git branch.

Now all these files are jumbled again.

Thank you @Rumperuu for renormalising the line ends f129f3a

Note that SVN does not have this issue.

@pewgeuges @Rumperuu is the escaping of quotation marks in the user input a new behaviour introduced by the refactoring here or has it historically been implemented?

It is a longlasting bug that I noticed only now. My bugreport is based on the plugin as it is deployed.

I'm not able to reproduce this on either main or wp-coding-standard--fresh.

Congratulations: You already fixed it, then! But how on main?

See here for my custom CSS (with quotation marks) and here the output. @pewgeuges have you definitely pulled the branch since 25c3f2f?

Yes I do always pull, but for the releases I needed to revert to the current plugin, and develop and test therein, until this branch has been merged. Then you will commit it to SVN updating trunk/, and then we’ll be able to use and release it with the next due release, as you planned, should any urgency arise before setting up the workflow in GitHub is completed.

Update: HTML elements within CSS are escaped; see here for custom CSS and here for output. This is also the case on main. I'm not sure if this is what @pewgeuges was referring to, however if it is then this is expected behaviour per the CSS spec.

Thank you for all the tests. That is not what I’m referring to, but html[dir="rtl"] becomes html[dir=\"rtl\"] in the dashboard and the header. It’s a matter of escaping quotes WRT MySQL; the bug is to not revert the escapement. It happens in all input fields, such as the reference container label.

@Rumperuu
Copy link
Collaborator Author

RTL compat bugfix v2.5.8 needs to be fast-tracked instead, see the latest support issue.

That's fine, 2.5.8 shouldn't be any bother to backport.

I did backport it, even prior to releasing it, but the line end issue has come back. Why is this branch using other line ends, and what can I do to address the problem?

Given this branch does not edit the style sheets, I pasted the modified ones over the existing ones. E.g. 12 minified style sheets were changed.

As of the readme.txt, I synced it alongside so it was handled the same way. Only footnotes.php was edited in the Git branch.

Now all these files are jumbled again.

Thank you @Rumperuu for renormalising the line ends f129f3a

Not to worry, however it happened f129f3a has fixed the ‘Files Changed’ tab view so we're good to go!

@pewgeuges @Rumperuu is the escaping of quotation marks in the user input a new behaviour introduced by the refactoring here or has it historically been implemented?

It is a longlasting bug that I noticed only now. My bugreport is based on the plugin as it is deployed.

I'm not able to reproduce this on either main or wp-coding-standard--fresh.

Congratulations: You already fixed it, then! But how on main?

How strange, I'm getting the bug on main now — not sure why I wasn't before. However, it does appear to be fixed on wp-coding-standard--fresh. I'll be quite honest, I haven't got a clue what fixed it, but it's always nice when you accidentally manage to fix something that you didn't even know was broken!

Anyway, if there are no more issues identified with this branch by 20:00 UK time I will merge it into main and then we can crack on with normal development again!

@pewgeuges
Copy link
Contributor

Thank you @Rumperuu for renormalising the line ends f129f3a

Not to worry, however it happened f129f3a has fixed the ‘Files Changed’ tab view so we're good to go!

If Footnotes’ GitHub main branch is now having other line ends than its SVN repo, should this be updated WRT line ends so that the issue won’t be replicated there?

Anyway, if there are no more issues identified with this branch by 20:00 UK time I will merge it into main and then we can crack on with normal development again!

Will you commit the updated main branch again, for the changes to be visible on SVN?  (WordPress don’t require to only commit release versions.)

@Rumperuu
Copy link
Collaborator Author

Rumperuu commented Feb 28, 2021

Thank you @Rumperuu for renormalising the line ends f129f3a

Not to worry, however it happened f129f3a has fixed the ‘Files Changed’ tab view so we're good to go!

If Footnotes’ GitHub main branch is now having other line ends than its SVN repo, should this be updated WRT line ends so that the issue won’t be replicated there?

I've not touched the SVN repo. since I cloned it to here, so the SVN codebase will have never received 8d0ebdb.

Anyway, if there are no more issues identified with this branch by 20:00 UK time I will merge it into main and then we can crack on with normal development again!

Will you commit the updated main branch again, for the changes to be visible on SVN?  (WordPress don’t require to only commit release versions.)

So once this PR is merged, we'll effectively have two codebases: footnotes here in main; and 'footnotes over on the SVN trunk. footnotes and 'footnotes will be feature-identical (as we've been backporting all changes from the SVN to here), and footnotes will be fully-standardised.

You have more experience with the SVN than I do, but am I correct in thinking that once the merge here is complete, we'll be able to just copy the files in main (excluding the development-only files like .gitignore, etc.) onto trunk and overwrite everything, so that both the Git and SVN repos contain the footnotes codebase? And then when we've got a new feature/bugfix ready to release on here, we can copy the files over again and make a release so that 2.5.9 (or whatever it might be called) publishes the standardisation changes too?

In which case (for my own benefit when I come to #30), the steps for an automated release workflow would be:

  1. run a build command on this Git repo that excludes development-only files, minifies files, updates the version number in readme.txt, etc.
  2. take the resulting files and copy them to the SVN trunk, overwriting whatever's there
  3. create a new tag on the SVN repo. from trunk with the new version number (svn cp trunk tags/<version number>)

Does that sound about right?

@Rumperuu Rumperuu merged commit 2e0b2f8 into main Feb 28, 2021
Development Workflow automation moved this from Reviewer approved to Done Feb 28, 2021
@Rumperuu Rumperuu deleted the wp-coding-standard--fresh branch February 28, 2021 20:03
@pewgeuges
Copy link
Contributor

If Footnotes’ GitHub main branch is now having other line ends than its SVN repo, should this be updated WRT line ends so that the issue won’t be replicated there?

I've not touched the SVN repo. since I cloned it to here, so the SVN codebase will have never received 8d0ebdb.

If we can be sure that SVN is EOL agnostic, it may not need it. But given GitHub is not, I doubt SVN is.

You are welcome to commit the EOL normalization and the standardised codebase. Hopefully SVN will then display a proper changeset, which I doubt it would without prior EOL normalisation.

Anyway, if there are no more issues identified with this branch by 20:00 UK time I will merge it into main and then we can crack on with normal development again!

Thank you for taking this step after you assessed all issues as solved!

You have more experience with the SVN than I do, but am I correct in thinking that once the merge here is complete, we'll be able to just copy the files in main (excluding the development-only files like .gitignore, etc.) onto trunk and overwrite everything,

I see a problem in doing so without previously normalising the EOL in SVN. Yesterday I tried to get hold of a method to normalize EOL but failed. You do have this method. Would you like to do the EOL fix? (And update trunk/ with main/?)

so that both the Git and SVN repos contain the footnotes codebase?

I think that will be great!

And then when we've got a new feature/bugfix ready to release on here, we can copy the files over again and make a release so that 2.5.9 (or whatever it might be called) publishes the standardisation changes too?

I would say yes, please go ahead.

In which case (for my own benefit when I come to #30), the steps for an automated release workflow would be:

  1. run a build command on this Git repo that excludes development-only files,

For use in my local sandbox I’m currently deleting these files and folders:

#!/bin/bash
# 2021-02-28T2217+0100
# gwp.sh
rm -rf GW/.git
rm -rf GW/.github
rm -rf GW/docs
rm -rf GW/_tools
rm -rf GW/.gitignore

Is that correct? The docs are twice as heavy as the whole codebase (1.6MB adding to 0.8MB), and as already reported they’re of limited use.

minifies files,

I think that this step may create more problems than it solves. The only file is settings.css, while all other stylesheets need to be concatenated first, as specified in development/csscat.sh, then these 12 files are minified, and then they’re deleted as pointless because the source files are shipped, along with the concatenation spec.

updates the version number in readme.txt, etc.

Beside the stable tag in readme.txt, there are two instances of the version number to be maintained in footnotes.php.

  1. take the resulting files and copy them to the SVN trunk, overwriting whatever's there

I don’t have enough experience to assess this process. I used to add or remove files one by one to/from version control in the SVN client, and commit. In the commit message, even double quotes enclosed in single quotes seem to need to be escaped, or SVN would fail to process a \n escape sequence.

  1. create a new tag on the SVN repo. from trunk with the new version number (svn cp trunk tags/<version number>)

That is correct, too.

Does that sound about right?

With the abovementioned caveats, yes, sounds good to me. Your automation effort is surely much appreciated, although I wouldn’t ask for so much.

Then there is also the pre-commit hook that you added support for.

@Rumperuu
Copy link
Collaborator Author

Rumperuu commented Mar 1, 2021

Learning Points from this PR

This took much longer than intended and broke my own suggestions in CONTRIBUTING.md about keeping changes small and atomic. Part of that was unavoidable due to the scope of the changes, but there are alternative ways that would have worked better and avoided blocking development for so long. For future reference, a better way of introducing this change would have been to

a) implement the automated code validation
b) performed the automatic fixes
c) suppressed all other warnings
d) merge that

This would have ensured that we got ~80% of the way to a standardised codebase in a single sweep, whilst allowing the more difficult manual changes to be dealt with one-by-one in separate, less-monolithic and easier-to-review PRs. Something will certainly bear in mind in future workflow implementation (+ other projects).

If Footnotes’ GitHub main branch is now having other line ends than its SVN repo, should this be updated WRT line ends so that the issue won’t be replicated there?

I've not touched the SVN repo. since I cloned it to here, so the SVN codebase will have never received 8d0ebdb.

If we can be sure that SVN is EOL agnostic, it may not need it. But given GitHub is not, I doubt SVN is.

You are welcome to commit the EOL normalization and the standardised codebase. Hopefully SVN will then display a proper changeset, which I doubt it would without prior EOL normalisation.

Done!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation duplicate This issue or pull request already exists wpcs-compliance WordPress Coding Standards conformance
3 participants