Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update easylist.txt #2154

Merged
merged 1 commit into from
Aug 11, 2021
Merged

Update easylist.txt #2154

merged 1 commit into from
Aug 11, 2021

Conversation

mjethani
Copy link
Contributor

easylist.txt and easyprivacy.txt are copied from ../adblocker/assets/… and now we're reading both files and concatenating them at runtime.

@@ -87,6 +87,11 @@ function loadLists() {
const filename = HOSTS_ONLY ? 'hosts.txt' : 'easylist.txt';
let content = fs.readFileSync(path.resolve(__dirname, filename), { encoding: 'utf-8' });

if (!HOSTS_ONLY) {
content += fs.readFileSync(path.resolve(__dirname, 'easyprivacy.txt'), { encoding: 'utf-8' })
.replace(/^\[Adblock\b.*\n/, '');
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we're concatenating lists we need to remove the "[Adblock Plus x.x]" at the top, lest it be interpreted as a filter.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By the way, the proper way to do this would be to pass each list on its own to the engine, but that would require changes in multiple places and might open another Pandora's box. I'd like to do it properly, but not in this patch.

@mjethani mjethani force-pushed the update-easylist branch 2 times, most recently from efbcb4a to 0b181e6 Compare August 11, 2021 14:36

// We still use old versions of EasyList and EasyPrivacy whereas current
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These transformations are no longer necessary.

// https://github.com/cliqz-oss/adblocker/discussions/2114#discussioncomment-1133958
content = content.replace(/(?!$.*)\bdomain=\|/gm, 'domain=');
if (!HOSTS_ONLY) {
content += fs.readFileSync(path.resolve(__dirname, 'easyprivacy.txt'), { encoding: 'utf-8' })
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why only in non-hosts mode?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The hosts.txt we are using already includes hosts for both ads and trackers (privacy). It's a different concept there.

@remusao remusao added the PR: Internal 🏠 Changes only affect internals label Aug 11, 2021
@remusao remusao merged commit 594f76a into ghostery:master Aug 11, 2021
@mjethani mjethani deleted the update-easylist branch August 11, 2021 23:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
PR: Internal 🏠 Changes only affect internals
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants