Remove resolve errors

[jayvdb]

List related PRs if any

• Subset of General tidy-up to allow matching rules with targets #18938

List related issues if any

• Regexps matching host that is not in targets #12297

[jayvdb]

(Rebased and added "Update ruleset-whitelist.csv" commit; no other changes)

[zoracon]

This has been stated on multiple PRs by the other maintainers but please split PR's by file. I understand there are many files that have issues, outdated hosts, etc. This will be helped along with the pending fetch test we plan to run. I also understand it may be frustrating to split ruleset fixes by file, but please trust that the maintainers will get to them in a timely manner.

[pipboy96]

@zoracon Can @jayvdb post the script that can produce these changes, and this script will be audited and applied by you?

I see the changes are likely manual. @jayvdb Please split this PR by file (I can help wit that if you don't know how to do that).

[jayvdb]

@pipboy96, @zoracon, I have been splitting these PRs. You can see the first batch at https://github.com/EFForg/https-everywhere/pulls/jayvdb . I have about 500 file changes pending, so I am primarily waiting for the maintainers to review those PRs before submitting the next batch of single-file PRs.

For a "script" which detects these problems, look no further than the https_everywhere_checker in this repository - the majority of the problems I am fixing are the files on the "whitelist" which bypass the rules in https_everywhere_checker. Removing the whitelist results in many errors -- actually fixing them instead of ignoring them is what is required, and is what these PRs are mostly about. There are a few valid reasons for whitelist entries, but them whitelist is mostly rulesets which have been not sane for years.

In addition to that, I have published a PyPI package which does rule analysis and reduction in https://github.com/jayvdb/https-everywhere-py/blob/master/https_everywhere/_rules.py . I then do manual further analysis of problems found in that automated analysis, in order to create hopefully optimal patches here. Obviously we have slightly different opinions about what shape optimal patches should take, so I am adjusting and hopefully we'll get more productive as we go along. I've also learnt more from the responses I have received from maintainers here (but there is lots of confusion also as different maintainers have wildly different responses sometimes ;-)), and my toolkit is also being improved.

[zoracon]

@jayvdb could you elaborate on the confusion of different responses from maintainers?

Right now the 4 challenges are dealing with outdated rulesets, outdated infrastructure, legacy code, and ongoing changes in the extension ecosystems. The maintainers and I are well aware of these issues and it's a balancing act that hopefully stabilizes more over the next few months. I still plan on running a fetch test, and that will yield a mass patch for rulesets that were completely disabled.

I'm not sure what bar of productivity you have in mind, but I want to remind contributors that the timeline is flexible for different aspects of the project and lately the focus has shifted to the upcoming fetch test, phasing out the problematic rulesets, and phasing out outdated conventions (like mixed content flag).

The maintainers and contributors (including you) have done a great job in the past year trying to make things more efficient and productive.

I am trying to think of ways to make reachable goals more transparent, and adjusting the timeline on the wiki soon to reflect that.

[jayvdb]

@zoracon , your own post highlights the most obvious and annoying case of maintainers giving mixed messages.

I still plan on running a fetch test, and that will yield a mass patch for rulesets

These are the issues that my PRs are fixing, but when I provide PRs with multiple fixes, the PRs are ignored and/or rejected until they are split, and the PRs are even labelled 'invalid'.

That leaves me feeling like I should just close all my PRs because the maintainers do not want help with this, because they can magically find and fix all these problems themselves, and they can bypass the 'rule' of one file per PR.

There are lots of other mixed messages I am getting, but they all seem to stem from that ingrained dismissive attitude, combined-with/caused-by the obvious fact that the maintainers are overworked which isnt surprising given the scale of the problem being tackled by this project.

[zoracon]

@jayvdb The fetch test is something that has only been run once every 2-3 years. Historically these involve mass patches but this time I want to break them up. The biggest patch will contain those that have been fully disabled. This is based off of the last fetch test PR #10538.

These are the issues that my PRs are fixing, but when I provide PRs with multiple fixes, the PRs are ignored and/or rejected until they are split, and the PRs are even labelled 'invalid'.

That leaves me feeling like I should just close all my PRs because the maintainers do not want help with this, because they can magically find and fix all these problems themselves, and they can bypass the 'rule' of one file per PR.

We have merged in about a dozen of your PRs, so it's clear that many of your contributions have been accepted. If you are upset about the few flagged as "invalid", then please consider the reasoning instead of assuming this is about dismissing help. No one feels that they can "magically", find and fix issues. I am the appointed lead on this project and what I do ask is trust that I'm not acting in the disinterest of the project based on my own ego.

I wouldn't say overworked, but during this time in many countries I ask patience and grace. We aren't operating under normal circumstances and expecting anything else would be unrealistic.