Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sort-of-pseudo-automated the Legacy list #6808

Closed
wants to merge 24 commits into from

Conversation

DandelionSprout
Copy link
Contributor

@DandelionSprout DandelionSprout commented Jan 10, 2020

As per DandelionSprout/adfilt#7 (comment) and some comments prior and afterwards.

Notes

I have zero idea how to make line = re.sub select text from multiple lines, which means that I am not able to remove !#if env_chromium- or !#if !env_firefox-dependant entries.

I also have zero idea how to make Python import local text files instead of URL sources, meaning that the script must be run twice to achieve the correct results. This is detailed in the README.md file.

The README.md file's instructions are based on downloading the file and running it locally with a file explorer and a terminal. If you guys were to figure out a way to do it in Git or Azure or whatever you guys use, you can write instructions for those methods beneath the existing instructions.

@mapx-
Copy link
Contributor

mapx- commented Jan 10, 2020

for nostif, nosiif etc see https://github.com/gorhill/uBlock/wiki/Resources-Library

@DandelionSprout
Copy link
Contributor Author

Okay, now I think I've got these things right in general.

@JustOff
Copy link
Contributor

JustOff commented Jan 10, 2020

I truly appreciate the effort to resolve the issue with legacy filters, but I'd like to warn against using the proposed solution in its current state, since I see at least two significant gaps:

  • The resulting legacy.txt contains invalid lines, such as, but not limited to:
 `google-analytics.com` in Peter Lowe's list
 Dan Pollock's, hpHosts, MVPS, Peter Lowe's
 `statcounter.com` in Peter Lowe's, MVPS, Dan Pollock's and hpHosts
 `2mdn.net` in Peter Lowe's
 `s0.2mdn.net` in MVPS, hpHosts, Dan Pollock's
 `adm.fwmrm.net` from hpHosts, MVPS, Dan Pollock's
 `mopub.com` in Peter Lowe's
 `safelinking.net` in hpHosts
 `outbrain.com` in hpHosts
 `flurry.com` in Peter Lowe's, hpHosts, MVPS
 `2mdn.net` in Peter Lowe's, `s0.2mdn.net` in hpHosts
 Peter Lowe's, hpHosts
 `yui.yahooapis.com` in hpHosts
  • The resulting legacy.txt always includes filters converted from all built-in uBlock filter lists (Annoyances, Badware risks, Privacy, Resource abuse, Unbreak), regardless of whether the user has enabled them or not.

@THEtomaso
Copy link

THEtomaso commented Jan 10, 2020

The resulting legacy.txt contains invalid lines

So, there a few bugs to weed out, but why would you try to convert anything from those hosts files?
They only contain network filters, none of which needs convertion!

--

The resulting legacy.txt always includes filters converted from all built-in uBlock filter lists

Oh, okiehsch specifically pointed out only to include rules from uBlock filters, and only the ones that doesn't work with uBO Legacy anymore:
https://github.com/DandelionSprout/adfilt/issues/7/#issuecomment-572953302

For the other built-in uBlock filter lists, additional Legacy versions are needed, like I suggested here:
https://github.com/DandelionSprout/adfilt/issues/7/#issuecomment-573006225
This will require a uBO Legacy extension update too, of course.

@DandelionSprout
Copy link
Contributor Author

I am aware that there's a handful of remnant comment lines that somehow haven't been properly !-ed, and I know of a fix for it, but I can't add it myself before 23:30 CET tonight due to handball spectating commitments.

As for which file(s) to use, that feels like something that I'm not the right guy to debate. I tried most of all to have it fit well into the existing legacy list's infrastructure.

@THEtomaso
Copy link

As for which file(s) to use, that feels like something that I'm not the right guy to debate.

What okiehsch ment is that uBlock filters - Legacy should only include rules converted from uBO's main filter list, and none from the specialized ones:

ublockorigin-filter4legacy

@JustOff
Copy link
Contributor

JustOff commented Jan 10, 2020

I'd suggest instead to consider the option to convert new shorthands back to legacy syntax directly in uBlock when retrieving filter lists.

Here is a simple hack doing so, based on rules taken from LegacyJSconverter.py.

@JustOff
Copy link
Contributor

JustOff commented Jan 14, 2020

Since the converter was added today to firefox-legacy-1.16.4.14, this PR has become obsolete.

Thanks again to @DandelionSprout for immediately offering help in resolving the legacy filter issue, as well as for the LegacyJSconverter.py that I used to create the converter embedded into uBlock.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants