New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DNS Client Must Be Disabled #56

Closed
Aerodog opened this Issue Jan 6, 2016 · 20 comments

Comments

Projects
None yet
7 participants
@Aerodog

Aerodog commented Jan 6, 2016

The size of the file requires the DNS Client to be disabled, otherwise the browser and ipconfig commands will appear to hang. This should be added to the readme.

(summary)
services.msc --> DNS Client (right click) --> properties --> startup "automatic" should be changed to "disabled" --> reboot

EDIT: I've noticed significant delays in webpage loading and some YouTube ads began appearing that I did not experience before. As a result, I've re-enabled the DNS Client and removed hosts-file.net from the folder for my own personal use. If there's an alternate solution to the problem, I'd be glad to hear it.

@StevenBlack

This comment has been minimized.

Show comment
Hide comment
@StevenBlack

StevenBlack Jan 6, 2016

Owner

@Aerodog which version of Windows are you running here?

I'm not a Windows user anymore so, if anyone has insight, please chime-in.

I'm mildly curious why a DNS cache would be sensitive to the hosts file. I would think these would be in separate layers. Anybody know?

Owner

StevenBlack commented Jan 6, 2016

@Aerodog which version of Windows are you running here?

I'm not a Windows user anymore so, if anyone has insight, please chime-in.

I'm mildly curious why a DNS cache would be sensitive to the hosts file. I would think these would be in separate layers. Anybody know?

@Aerodog

This comment has been minimized.

Show comment
Hide comment
@Aerodog

Aerodog Jan 6, 2016

I'm using Windows 10, but this is a well-known Windows-only issue on the hosts-file.net forums.

http://forum.hosts-file.net/viewtopic.php?f=6&t=2405

http://forum.hosts-file.net/viewtopic.php?f=6&t=4753

I haven't looked much into the reason. In the second link, a staff member of the site mentioned, "You need to disable the DNS Client service when using a large file, if not using the optimized one." Information regarding installation on Windows as well as the "optimized" file can be found here:
http://hosts-file.net/?s=Download

EDIT: http://hosts-file.net/download/hosts-optimized.txt

Aerodog commented Jan 6, 2016

I'm using Windows 10, but this is a well-known Windows-only issue on the hosts-file.net forums.

http://forum.hosts-file.net/viewtopic.php?f=6&t=2405

http://forum.hosts-file.net/viewtopic.php?f=6&t=4753

I haven't looked much into the reason. In the second link, a staff member of the site mentioned, "You need to disable the DNS Client service when using a large file, if not using the optimized one." Information regarding installation on Windows as well as the "optimized" file can be found here:
http://hosts-file.net/?s=Download

EDIT: http://hosts-file.net/download/hosts-optimized.txt

@lencc

This comment has been minimized.

Show comment
Hide comment
@lencc

lencc Jan 14, 2016

I had this problem on Windows 8 as well. Internet didn't work anymore even after a restart. I couldn't replace hosts file with the older ("smaller") one, so I had to boot into safe mode to replace it with the old one (I made a backup hosts file).

I suppose the DNS Client service has to remain disabled (and not only temporarily disabled - while I run flushdns command) in order for the current hosts file to work properly in Windows. Are there any consequences for Windows users with disabled DNS Client service?

lencc commented Jan 14, 2016

I had this problem on Windows 8 as well. Internet didn't work anymore even after a restart. I couldn't replace hosts file with the older ("smaller") one, so I had to boot into safe mode to replace it with the old one (I made a backup hosts file).

I suppose the DNS Client service has to remain disabled (and not only temporarily disabled - while I run flushdns command) in order for the current hosts file to work properly in Windows. Are there any consequences for Windows users with disabled DNS Client service?

@StevenBlack

This comment has been minimized.

Show comment
Hide comment
@StevenBlack

StevenBlack Jan 15, 2016

Owner

I just removed hosts-file.net from the repo.

One reason is it was creating problems such as this one.

But the main reason is, that hosts file doesn't have the curation activity I expect.

Better now?

Owner

StevenBlack commented Jan 15, 2016

I just removed hosts-file.net from the repo.

One reason is it was creating problems such as this one.

But the main reason is, that hosts file doesn't have the curation activity I expect.

Better now?

@lencc

This comment has been minimized.

Show comment
Hide comment
@lencc

lencc Jan 16, 2016

Great, thanks! It's much better now, because the "huge" hosts file (together with disabled DNS Client service) actually decreased the internet speed significantly, as far as I tested it in Windows.

I have one more suggestion to further reduce the size of hosts file: remove the following three lists from the hosts, because they are all disabled anyway:

  • "Windows10"
  • "maybe-ads"
  • "maybe-spy" (except the website media.fastclick.net beacuse this one is not disabled)

You could then create additional (secondary) hosts file with the upper lists only. All the entries could then be enabled. Therefore advanced users could manually add them to the "original" hosts file. All of the removed entries (300,000 websites from hosts-file.net) could also be located in this secondary hosts file.

Furher, all the "Acknowledgements" could be removed from the hosts file (they are located just before the "Hosts contributed by Steven Black"). Acknowledgements could be instead written in the file readme.md (or anywhere else).

lencc commented Jan 16, 2016

Great, thanks! It's much better now, because the "huge" hosts file (together with disabled DNS Client service) actually decreased the internet speed significantly, as far as I tested it in Windows.

I have one more suggestion to further reduce the size of hosts file: remove the following three lists from the hosts, because they are all disabled anyway:

  • "Windows10"
  • "maybe-ads"
  • "maybe-spy" (except the website media.fastclick.net beacuse this one is not disabled)

You could then create additional (secondary) hosts file with the upper lists only. All the entries could then be enabled. Therefore advanced users could manually add them to the "original" hosts file. All of the removed entries (300,000 websites from hosts-file.net) could also be located in this secondary hosts file.

Furher, all the "Acknowledgements" could be removed from the hosts file (they are located just before the "Hosts contributed by Steven Black"). Acknowledgements could be instead written in the file readme.md (or anywhere else).

@StevenBlack

This comment has been minimized.

Show comment
Hide comment
@StevenBlack

StevenBlack Jan 23, 2016

Owner

Closing.

Owner

StevenBlack commented Jan 23, 2016

Closing.

@Gitoffthelawn

This comment has been minimized.

Show comment
Hide comment
@Gitoffthelawn

Gitoffthelawn Jan 26, 2016

@StevenBlack Hi Steven. You mentioned "I just removed hosts-file.net from the repo ... the main reason is, that hosts file doesn't have the curation activity I expect.".

Are you referring to hosts-file.net/ad_servers.txt ?

If so, I've noticed that it seems to have more frequent updates than many other hosts. AdAway uses it as one of its 4 defaults hosts files, and that file is the one with some of the most frequent updates.

That's just what I've noticed... what did you notice?

Gitoffthelawn commented Jan 26, 2016

@StevenBlack Hi Steven. You mentioned "I just removed hosts-file.net from the repo ... the main reason is, that hosts file doesn't have the curation activity I expect.".

Are you referring to hosts-file.net/ad_servers.txt ?

If so, I've noticed that it seems to have more frequent updates than many other hosts. AdAway uses it as one of its 4 defaults hosts files, and that file is the one with some of the most frequent updates.

That's just what I've noticed... what did you notice?

@Laicure

This comment has been minimized.

Show comment
Hide comment
@Laicure

Laicure Jan 26, 2016

Contributor

@Gitoffthelawn I think @StevenBlack is referring to the 11MB+ Host file which has currently 336,509 entries that breaks certain parsing or something like it did on my mobile device. Well, that ad_servers.txt is quite a good list though~ ☺️

Contributor

Laicure commented Jan 26, 2016

@Gitoffthelawn I think @StevenBlack is referring to the 11MB+ Host file which has currently 336,509 entries that breaks certain parsing or something like it did on my mobile device. Well, that ad_servers.txt is quite a good list though~ ☺️

@FadeMind

This comment has been minimized.

Show comment
Hide comment
@FadeMind

FadeMind Jan 26, 2016

Contributor

@StevenBlack @Gitoffthelawn after merging ad_servers.txt hosts file have:
68,118 entries.
Size: 2.3 MB

Contributor

FadeMind commented Jan 26, 2016

@StevenBlack @Gitoffthelawn after merging ad_servers.txt hosts file have:
68,118 entries.
Size: 2.3 MB

@lencc

This comment has been minimized.

Show comment
Hide comment
@lencc

lencc Jan 26, 2016

@FadeMind does this 2.3 MB merged hosts file work also on Windows without the need for disabling DNS Client service?

lencc commented Jan 26, 2016

@FadeMind does this 2.3 MB merged hosts file work also on Windows without the need for disabling DNS Client service?

@FadeMind

This comment has been minimized.

Show comment
Hide comment
@FadeMind

FadeMind Jan 26, 2016

Contributor

@lencc IDK. See: http://winhelp2002.mvps.org/hostswin7.htm and
"The below does not apply to Windows 10 ... seems Microsoft has finally corrected the issue ...

Before you can install a custom HOSTS file in Win8, you will need to make a change in Windows Defender ... otherwise Defender will automatically remove your custom file and replace it with the default from Microsoft."
from: http://winhelp2002.mvps.org/hostswin8.htm

I using Arch Linux, and 23 MB hosts file don't affect me... but this is extreme customizing hosts file.

Contributor

FadeMind commented Jan 26, 2016

@lencc IDK. See: http://winhelp2002.mvps.org/hostswin7.htm and
"The below does not apply to Windows 10 ... seems Microsoft has finally corrected the issue ...

Before you can install a custom HOSTS file in Win8, you will need to make a change in Windows Defender ... otherwise Defender will automatically remove your custom file and replace it with the default from Microsoft."
from: http://winhelp2002.mvps.org/hostswin8.htm

I using Arch Linux, and 23 MB hosts file don't affect me... but this is extreme customizing hosts file.

@qutorial

This comment has been minimized.

Show comment
Hide comment
@qutorial

qutorial Jan 26, 2016

@FadeMind as you do already use a huge hosts file, you might want to know about why it is better to use dnsmasq for this, and a fork of this tool, which actually does it:
https://molotnikov.de/dnsmasq
https://github.com/qutorial/angryhostsfile
I called it differently from simply "hosts", it also serves a different purpose, then the hosts file here: it blocks A LOT more. So some more things will stop working time-to-time.
In this sense the hosts file discussed here is more user-friendly

qutorial commented Jan 26, 2016

@FadeMind as you do already use a huge hosts file, you might want to know about why it is better to use dnsmasq for this, and a fork of this tool, which actually does it:
https://molotnikov.de/dnsmasq
https://github.com/qutorial/angryhostsfile
I called it differently from simply "hosts", it also serves a different purpose, then the hosts file here: it blocks A LOT more. So some more things will stop working time-to-time.
In this sense the hosts file discussed here is more user-friendly

@StevenBlack

This comment has been minimized.

Show comment
Hide comment
@StevenBlack

StevenBlack Jan 26, 2016

Owner

I have some feelings about all this.

What matters most to me is that our sources be actively curated so new threats are identified quickly, but also that false-positives are removed quickly.

When I was watching closely, I didn't see much active curation at hosts-file.net. I suspect one way to get an 11 mb hosts file is always cumulate, and rarely trim. Maybe hosts-file.net has scaled beyond the size where effective curation is even possible?

I think assembling all the possible sources, and producing a very large (monster) hosts file, is one way to do this. This will definitely protect you.

But I'm more interested in a solution that strikes a better balance.

I don't mind seeing the odd ad. But I don't ever want to see malware. I suspect those who first catch the sources of malware are those who actively curate their hosts files most frequently. This is just a hunch.

Owner

StevenBlack commented Jan 26, 2016

I have some feelings about all this.

What matters most to me is that our sources be actively curated so new threats are identified quickly, but also that false-positives are removed quickly.

When I was watching closely, I didn't see much active curation at hosts-file.net. I suspect one way to get an 11 mb hosts file is always cumulate, and rarely trim. Maybe hosts-file.net has scaled beyond the size where effective curation is even possible?

I think assembling all the possible sources, and producing a very large (monster) hosts file, is one way to do this. This will definitely protect you.

But I'm more interested in a solution that strikes a better balance.

I don't mind seeing the odd ad. But I don't ever want to see malware. I suspect those who first catch the sources of malware are those who actively curate their hosts files most frequently. This is just a hunch.

@qutorial

This comment has been minimized.

Show comment
Hide comment
@qutorial

qutorial Jan 26, 2016

I fully agree with you, and understand the philosophy of the hosts file you provide here.
This is why I decided to finally fork: the angry version is more paranoid and is for those who are a bit more paranoid.

For example, I do mind odd ads. Maldvertising is getting popular. I do mind trackers as well and they are not blocked enough by this hosts file. Not blocked malicious subdomains is another very serious disadvantage of the block-by-hostsfile approach vs. dnsmasq.

Now the hosts file accumulated from many source contains ~430K hosts. It blocks everything blocked by this hosts file and many hosts more. The mechanism of whitelisting and logs of dnsmasq or a browser console are there to find and unblock something occasionally wrongly blocked or useful (despite of some "bad" properties) at the moment.

In my experience, it is possible to use the web most of the time without any problems. Sometimes though, when DNS-blocking is on the way, I have to whitelist. In this sense the more paranoid approach is not the same user-friendly and comfortable.

qutorial commented Jan 26, 2016

I fully agree with you, and understand the philosophy of the hosts file you provide here.
This is why I decided to finally fork: the angry version is more paranoid and is for those who are a bit more paranoid.

For example, I do mind odd ads. Maldvertising is getting popular. I do mind trackers as well and they are not blocked enough by this hosts file. Not blocked malicious subdomains is another very serious disadvantage of the block-by-hostsfile approach vs. dnsmasq.

Now the hosts file accumulated from many source contains ~430K hosts. It blocks everything blocked by this hosts file and many hosts more. The mechanism of whitelisting and logs of dnsmasq or a browser console are there to find and unblock something occasionally wrongly blocked or useful (despite of some "bad" properties) at the moment.

In my experience, it is possible to use the web most of the time without any problems. Sometimes though, when DNS-blocking is on the way, I have to whitelist. In this sense the more paranoid approach is not the same user-friendly and comfortable.

@lencc

This comment has been minimized.

Show comment
Hide comment
@lencc

lencc Jan 26, 2016

@FadeMind could you upload this merged file online (i.e. merged current StevenBlack hosts + current ad_servers.txt hosts), so I can test it?

As regards Windows Defender and automatic replacement of custom hosts file: it is interesting that I didn't have such problems on Windows 8, despite I have always been using Windows Defender. However I have upgraded to Windows 10 recently. :) Thanks anyway for the information.

lencc commented Jan 26, 2016

@FadeMind could you upload this merged file online (i.e. merged current StevenBlack hosts + current ad_servers.txt hosts), so I can test it?

As regards Windows Defender and automatic replacement of custom hosts file: it is interesting that I didn't have such problems on Windows 8, despite I have always been using Windows Defender. However I have upgraded to Windows 10 recently. :) Thanks anyway for the information.

@FadeMind

This comment has been minimized.

Show comment
Hide comment
@FadeMind
Contributor

FadeMind commented Jan 26, 2016

@lencc

This comment has been minimized.

Show comment
Hide comment
@lencc

lencc Jan 26, 2016

Great, thank you! It works fine in Windows 10 (with enabled DNS Client).

lencc commented Jan 26, 2016

Great, thank you! It works fine in Windows 10 (with enabled DNS Client).

@Gitoffthelawn

This comment has been minimized.

Show comment
Hide comment
@Gitoffthelawn

Gitoffthelawn Jan 27, 2016

First, @Laicure thank you for the details you provided. @StevenBlack can you confirm Laicure's guess as to which list you were referring? I have a hunch Laicure is correct and that you and I were referring to different lists from the same site.

I think this thread contains an excellent and thoughtful discussion. Everyone has a different level of precisely what they want to be blocked, but in general, several different broad categories can be defined which will satisfy an overwhelming majority of people.

I believe it is in the best interest of all these projects to make their goals clearly visible in a README. Steven's post above is a great start to such a document:

  1. What is the overall philosophy
  2. What does it block
  3. What does it not block
  4. What are the trade-offs

Gitoffthelawn commented Jan 27, 2016

First, @Laicure thank you for the details you provided. @StevenBlack can you confirm Laicure's guess as to which list you were referring? I have a hunch Laicure is correct and that you and I were referring to different lists from the same site.

I think this thread contains an excellent and thoughtful discussion. Everyone has a different level of precisely what they want to be blocked, but in general, several different broad categories can be defined which will satisfy an overwhelming majority of people.

I believe it is in the best interest of all these projects to make their goals clearly visible in a README. Steven's post above is a great start to such a document:

  1. What is the overall philosophy
  2. What does it block
  3. What does it not block
  4. What are the trade-offs
@StevenBlack

This comment has been minimized.

Show comment
Hide comment
@StevenBlack

StevenBlack Feb 2, 2016

Owner

@Gitoffthelawn thank you again for that suggestion. I've updated the readme to clarify the goals of the repo.

Owner

StevenBlack commented Feb 2, 2016

@Gitoffthelawn thank you again for that suggestion. I've updated the readme to clarify the goals of the repo.

@Gitoffthelawn

This comment has been minimized.

Show comment
Hide comment
@Gitoffthelawn

Gitoffthelawn Feb 4, 2016

@StevenBlack You're welcome. Best regards to you.

Gitoffthelawn commented Feb 4, 2016

@StevenBlack You're welcome. Best regards to you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment