Skip to content
This repository has been archived by the owner. It is now read-only.

Add "Malware Domain List" and "Malware domains" to malware_lists #142

Closed
wants to merge 1 commit into from

Conversation

@rht
Copy link
Contributor

rht commented Nov 28, 2018

There could be perf concerns for having too big of a list, but these
additional sources should be available as an option for those who prefer
the blocking to be as comprehensive as uBlock Origin.
Based on https://github.com/gorhill/uBlock/blob/e37cc238a772b1e51d9b2e2ea7ab33b14a12dc64/assets/assets.json#L171-L194.

@rht rht force-pushed the rht:ublock-malware-parity branch 2 times, most recently from cf8ceed to dd00ba7 Nov 28, 2018
There could be perf concerns for having too big of a list, but these
additional sources should be available as an option for those who prefer
the blocking to be as comprehensive as uBlock Origin.
Based on https://github.com/gorhill/uBlock/blob/e37cc238a772b1e51d9b2e2ea7ab33b14a12dc64/assets/assets.json#L171-L194.
@rht rht force-pushed the rht:ublock-malware-parity branch from dd00ba7 to 9a15999 Nov 28, 2018
@rht
Copy link
Contributor Author

rht commented Nov 28, 2018

(How do I disable the 2 newly added sources?)

Copy link
Member

bbondy left a comment

Note that these rules should be domain anchored filters.
So starting with || at the start.
That way they are just a simple insertion into a hashset which is fast.

Please see the hack we do for the similarly formatted simple_malware.txt list though:
git grep FBB430E8-3910-4761-9373-840FC3B43FF2

And then you can apply it for these lists too:

const getListFilterFunction = (uuid) => {
  if (uuid === 'FBB430E8-3910-4761-9373-840FC3B43FF2') {
    return (input) => input.split('\n').slice(4)
      .map((line) => `||${line}`).join('\n')
  }
  return undefined
}
@bbondy bbondy requested review from fmarier and diracdeltas Feb 27, 2019
@bbondy
Copy link
Member

bbondy commented Feb 27, 2019

@rht Sorry for not noticing this PR sooner. If you want an invite to our Slack in a guest channel please shoot me an email at brian@brave.com and then you can ping for any PR that takes too long.
Sorry if you're already there and I'm not matching up your username.

@diracdeltas
Copy link
Member

diracdeltas commented Feb 27, 2019

does this mean domains from these lists will be blocked by default? (or is it opt-in?)

i think we want to make sure that these list maintainers have some reasonable guards against false positives, otherwise legit sites could end up being blocked. for instance https://www.malwaredomains.com/?p=4168

BTW the spam404 list was originally because we didn't have safebrowsing integrated; now that we do, it may no longer be necessary

"https://gitcdn.xyz/repo/NanoMeow/MDLMirror/master/hosts.txt",
"Malware Domain List",
{},
"http://www.malwaredomainlist.com/",

This comment has been minimized.

Copy link
@fmarier

fmarier Feb 28, 2019

Member

Their website used to show list updates but the last one is from 2017-12-04. Their forum looks abandoned too. The last post was 2018-05-09 and it was a report of a malware site by a user.

The download link included in this PR ultimately points to this fairly new GitHub repo. Looking at that repo, the first commit which imported the list, was on 2019-01-04. The only other commit which has changed the list since then was on 2019-02-20. It looks like it's still maintained, but it's not particularly active.

@@ -27,6 +27,22 @@ const std::vector<FilterList> malware_lists = {
"https://disconnect.me/",
"",
""
}), FilterList({
"E5484F0C-F35C-11E8-8EB2-F2801F1B9FD1",
"https://gitcdn.xyz/repo/NanoMeow/MDLMirror/master/hosts.txt",

This comment has been minimized.

Copy link
@fmarier

fmarier Feb 28, 2019

Member

The underlying repo also includes a list in the adblock format and so we should use this URL instead:
https://github.com/NanoMeow/MDLMirror/raw/master/filter.txt

"https://mirror.cedia.org.ec/malwaredomains/justdomains",
"Malware domains",
{},
"https://www.malwaredomains.com/",

This comment has been minimized.

Copy link
@fmarier

fmarier Feb 28, 2019

Member

Many of the mirrors they list on their website are out-of-date but the one used in this pull request seems seems to be fresh (last update 2019-02-20).

They used to post about list updates and false positives on their homepage but the last update post was on 2018-03-08 and the last false positive announcement was on 2018-06-08.

That said, they appear to still be adding and removing domains from their list.

@bbondy
Copy link
Member

bbondy commented Feb 28, 2019

I think we're only using these lists in mobile today. If at all.

@fmarier
Copy link
Member

fmarier commented Feb 28, 2019

I think we're only using these lists in mobile today. If at all.

@diracdeltas and I tested this yesterday and these lists are currently used only on iOS. I opened brave/browser-android-tabs#1212 for Android.

@diracdeltas
Copy link
Member

diracdeltas commented Mar 26, 2019

@rht On second thought, we decided not to add any more third-party lists for malware protection and may even filter out some of the existing entries to reduce the risk of false positives. We may revisit this decision in the future. Thanks anyway for the PR though!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants
You can’t perform that action at this time.