Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MalwareDomainList.com is stale, no update since February; suggest removing or set it as a optional list. #1044

Closed
opav opened this issue Sep 9, 2019 · 11 comments

Comments

@opav
Copy link

opav commented Sep 9, 2019

MalwareDomainList.com list is stale, no update since February. Even if it's updated, the fact that malware domains are ephemeral in nature, meaning once they are publicly known or listed, it is more or less dead. Unless one have access to private (expensive) listings, subscribing to public malware domain listings is quite pointless.

And having such a huge list is heavy on resources.

So I suggest its removal from the Unified list and be made as a optional list for those who still really wants it.

@ndrwy
Copy link

ndrwy commented Sep 9, 2019

There's also adaway.org which hasn't been updated in 2 years by its author: https://github.com/StevenBlack/hosts/tree/master/data / https://github.com/AdAway/adaway.github.io/commits/master/hosts.txt

But I'm not aware of any suitable "successor" and this may not be an urgent issue if people haven't submitted false positives for domains on the above list en masse. Found a related discussion: AdAway/adaway.github.io#13 (the list is https://github.com/jawz101/MobileAdTrackers)

There's also UncheckyAds (last update 1 year ago), but the author is still active and the list is relatively small and specific: https://github.com/FadeMind/hosts.extras/commits/master

@dnmTX
Copy link
Contributor

dnmTX commented Sep 9, 2019

@ndrwy as you pointed out already they're in process of merging different,more frequently updated lists.So,give it time.

@StevenBlack
Copy link
Owner

Hi @opav I agree, this list is getting stale, and yes, it's been awhile since its last update.

I'm inclined to keep this list awhile longer.

With all lists, activity tends to come in waves. All lists have periods where they are more dynamic than others. It doesn't mean a quiet list isn't any good.

Let's revisit this in a few months.

@durablenapkin
Copy link

Just to follow up on this, AdAway merged jawz101/MobileAdTrackers yesterday! :) AdAway/adaway.github.io#13

@StevenBlack
Copy link
Owner

So here are some stats.

  • The last version of malwaredomainlist contains 1,105 domains.
  • Of those, 246 domains are found within other curated sources in our default list — 22% of malwaredomainlist is covered.
  • Therefore malwaredomainlist brings 859 unique domains to our present list.

@lightswitch05
Copy link
Contributor

I actually have a little script to do a diff of hosts files. Doing a diff of a847730 using the full hosts and one generated without malwaredomainlist gives me these stats in case anyone is interested

Number of unique host entries:
hosts                         50343
mdl-hosts                     1106

Number of unique hosts not in the other list:
hosts                         50097 (99.51%)
mdl-hosts                     860 (77.76%)

Full diff can be viewed here. Hosts that are on both lists are excluded. Left side are unique hosts in the base list minus malwaredomainlist and right right side are hosts that are only in malwaredomainlist.

I would be really interested to see that list of 860 hosts ran through PyFunceble to see how many hosts can be considered up and reachable.

@lightswitch05
Copy link
Contributor

In case @funilrys has an interest in setting up a PyFunceble run, here are the hosts that are in Malware Domain List only: https://gist.github.com/lightswitch05/7efdffd232b2dde88f13b607e3c80c10

@dnmTX
Copy link
Contributor

dnmTX commented Jan 20, 2020

@lightswitch05 Malware Domain List is already included in Ultimate-Hosts-Blacklist ->Link for a while now. Unique domains are not filtered though but you can at least check how many are ACTIVE and compare it with your list.

@lightswitch05
Copy link
Contributor

I should have known it was already setup. Thanks for the info @dnmTX. I’ll run a diff against the active list and the 860 hosts not already included in other lists and report my findings

@lightswitch05
Copy link
Contributor

So, of the 860 hosts not covered in other lists, 378 are considered active per PyFunceble. Active hosts that are only on Malware Domain List

spirillen added a commit to mypdns/matrix that referenced this issue Jan 22, 2020
…one and

third party ~Malicious hosts sources.

My believe is therefore we should be able to reduce the necessary entries for
this second level domain to be blocked by wildcard.

Of curse, trying to add this report to Gilab failed by there SpyWare

<https://user-images.githubusercontent.com/5105020/72882541-16817300-3d03-11ea-9631-ae4f6e1e4ce8.png>

See more about this big issue at:
<​https://gitlab.com/my-privacy-dns/matrix/matrix/issues/74> and
<​https://gitlab.com/gitlab-com/support-forum/issues/4521>

Closes ​StevenBlack/hosts#1044
Closing https://www.mypdns.org/ticket/144
@StevenBlack
Copy link
Owner

MalwareDomainList updated yesterday, or thereabouts.

Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants