Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

List of Windows 10 tracking / telemetry / ads hosts? #155

Closed
Gitoffthelawn opened this issue Jul 25, 2016 · 42 comments
Closed

List of Windows 10 tracking / telemetry / ads hosts? #155

Gitoffthelawn opened this issue Jul 25, 2016 · 42 comments

Comments

@Gitoffthelawn
Copy link

@Gitoffthelawn Gitoffthelawn commented Jul 25, 2016

I want to do some research into this whole Windows 10 tracking / telemetry / ads fiasco.

To give me a head start, is there a list of known hosts used for those purposes?

IOW, not a list of hosts for used by every advertising / tracking provider in the world: just a specific list for hosts used by Windows 10 for tracking / telemetry / ads.

@berrythesoftwarecodeprogrammar

This comment has been minimized.

@berrythesoftwarecodeprogrammar

This comment has been minimized.

Copy link

@berrythesoftwarecodeprogrammar berrythesoftwarecodeprogrammar commented Jul 31, 2016

i also just came across this so im going to try it https://www.safer-networking.org/spybot-anti-beacon/

@berrythesoftwarecodeprogrammar

This comment has been minimized.

Copy link

@berrythesoftwarecodeprogrammar berrythesoftwarecodeprogrammar commented Jul 31, 2016

seems like a good program, added some extra protection. the only extra host it added was:

0.0.0.0 choice.microsoft.com.nstac.net
@Gitoffthelawn

This comment has been minimized.

Copy link
Author

@Gitoffthelawn Gitoffthelawn commented Aug 1, 2016

@berrythesoftwarecodeprogrammar Thanks so much Berry. Are there any curated lists that are being kept updated for this purpose? With the forced-updates design of Windows 10, MS can add/modify which hosts all Windows 10 systems connect to at any time (or even have some systems connect to certain hosts, while other systems connect to different hosts).

For the spybot anti beacon software you mentioned, did it just include that one host, or was that just the only host that was not included in this repo?

@berrythesoftwarecodeprogrammar

This comment has been minimized.

Copy link

@berrythesoftwarecodeprogrammar berrythesoftwarecodeprogrammar commented Aug 2, 2016

@Gitoffthelawn that was just the only one not included in the other list i pasted. which was a list i think i extracted from the Destroy Windows Spying program, or something similar

@berrythesoftwarecodeprogrammar

This comment has been minimized.

Copy link

@berrythesoftwarecodeprogrammar berrythesoftwarecodeprogrammar commented Aug 2, 2016

There are just a variety of programs which aim to stop windows spying, especially W10 related stuff. A bunch of them bundle hosts entries with them and I put together the ones which I used. I trust the Spybot team, since I've used their software for a long time and I think they would have the most reliable and up to date list of hosts.

@berrythesoftwarecodeprogrammar

This comment has been minimized.

Copy link

@berrythesoftwarecodeprogrammar berrythesoftwarecodeprogrammar commented Aug 2, 2016

Here are the previous links and more, for anyone who hasn't seen the other thread:

https://www.privacytools.io/#win10 -- Links to various tools and information about Windows 10 spying
https://fix10.isleaked.com/ -- Guide to disabling most of the bad features in Windows 10
https://fix10.isleaked.com/oldwindows.html -- Guide to removing bad updates from Windows 7/8
https://www.safer-networking.org/spybot-anti-beacon/ -- Software to stop telemetry in Windows 7/8/10
http://dws.wzor.net/ -- Software to stop and/or remove unwanted features of Windows 7/8/10
http://ultimateoutsider.com/downloads/ -- GWX Control Panel; Prevent Windows 7/8 from updating to 10

I use all of the software above since they each have their own special features.

@berrythesoftwarecodeprogrammar

This comment has been minimized.

Copy link

@berrythesoftwarecodeprogrammar berrythesoftwarecodeprogrammar commented Aug 2, 2016

I use the tyzbit hosts file and I have these Microsoft/Windows related entries in my myhosts file:

0.0.0.0 a-0001.a-msedge.net
0.0.0.0 a.ads1.msn.com
0.0.0.0 a.ads2.msn.com
0.0.0.0 ad.doubleclick.net
0.0.0.0 adnexus.net
0.0.0.0 adnxs.com
0.0.0.0 ads.msn.com
0.0.0.0 ads1.msads.net
0.0.0.0 ads1.msn.com
0.0.0.0 az361816.vo.msecnd.net
0.0.0.0 az512334.vo.msecnd.net
0.0.0.0 ca.telemetry.microsoft.com
0.0.0.0 cache.datamart.windows.com
0.0.0.0 choice.microsoft.com
0.0.0.0 choice.microsoft.com.nsatc.net
0.0.0.0 choice.microsoft.com.nstac.net
0.0.0.0 compatexchange.cloudapp.net
0.0.0.0 corp.sts.microsoft.com
0.0.0.0 corpext.msitadfs.glbdns2.microsoft.com
0.0.0.0 cs1.wpc.v0cdn.net
0.0.0.0 db3wns2011111.wns.windows.com
0.0.0.0 df.telemetry.microsoft.com
0.0.0.0 diagnostics.support.microsoft.com
0.0.0.0 fe2.update.microsoft.com.akadns.net
0.0.0.0 fe3.delivery.dsp.mp.microsoft.com.nsatc.net
0.0.0.0 feedback.microsoft-hohm.com
0.0.0.0 feedback.search.microsoft.com
0.0.0.0 feedback.windows.com
0.0.0.0 i1.services.social.microsoft.com
0.0.0.0 i1.services.social.microsoft.com.nsatc.net
0.0.0.0 msnbot-207-46-194-33.search.msn.com
0.0.0.0 oca.telemetry.microsoft.com
0.0.0.0 oca.telemetry.microsoft.com.nsatc.net
0.0.0.0 pre.footprintpredict.com
0.0.0.0 preview.msn.com
0.0.0.0 rad.msn.com
0.0.0.0 redir.metaservices.microsoft.com
0.0.0.0 reports.wes.df.telemetry.microsoft.com
0.0.0.0 s0.2mdn.net
0.0.0.0 services.wes.df.telemetry.microsoft.com
0.0.0.0 settings-sandbox.data.microsoft.com
0.0.0.0 settings-win.data.microsoft.com
0.0.0.0 settings.data.microsof.com
0.0.0.0 sls.update.microsoft.com.akadns.net
0.0.0.0 spynet2.microsoft.com
0.0.0.0 spynetalt.microsoft.com
0.0.0.0 sqm.df.telemetry.microsoft.com
0.0.0.0 sqm.telemetry.microsoft.com
0.0.0.0 sqm.telemetry.microsoft.com.nsatc.net
0.0.0.0 ssw.live.com
0.0.0.0 statsfe1.ws.microsoft.com
0.0.0.0 statsfe2.update.microsoft.com.akadns.net
0.0.0.0 statsfe2.ws.microsoft.com
0.0.0.0 survey.watson.microsoft.com
0.0.0.0 telecommand.telemetry.microsoft.com
0.0.0.0 telecommand.telemetry.microsoft.com.nsatc.net
0.0.0.0 telemetry.appex.bing.net
0.0.0.0 telemetry.microsoft.com
0.0.0.0 telemetry.urs.microsoft.com
0.0.0.0 v10.vortex-win.data.microsoft.com
0.0.0.0 view.atdmt.com
0.0.0.0 vortex-sandbox.data.microsoft.com
0.0.0.0 vortex-win.data.microsoft.com
0.0.0.0 vortex.data.microsoft.com
0.0.0.0 watson.live.com
0.0.0.0 watson.microsoft.com
0.0.0.0 watson.ppe.telemetry.microsoft.com
0.0.0.0 watson.telemetry.microsoft.com
0.0.0.0 watson.telemetry.microsoft.com.nsatc.net
0.0.0.0 wes.df.telemetry.microsoft.com
0.0.0.0 win10.ipv6.microsoft.com

(Extracted fromthe hosts files which those programs install, since I use this script to create my hosts file and don't want to have to run external programs everytime I update my hosts)

@CHEF-KOCH

This comment has been minimized.

Copy link

@CHEF-KOCH CHEF-KOCH commented Aug 2, 2016

The mentioned stuff is an relict from earlier Windows previews. You only need one source, which constalty gets updated, it's this one. The mentioned hosts entries are really used (mostly because Windows Store or search). All others aren't in usage anymore.

@StevenBlack

This comment has been minimized.

Copy link
Owner

@StevenBlack StevenBlack commented Aug 2, 2016

Well, we already have someonewhocares.org.

@Gitoffthelawn

This comment has been minimized.

Copy link
Author

@Gitoffthelawn Gitoffthelawn commented Aug 3, 2016

@CHEF-KOCH Thanks, but that list contains much more than just MS Win10 stuff... well, at least I hope so! 😉

@07416

This comment has been minimized.

Copy link

@07416 07416 commented Sep 6, 2016

Domains added by DisableWinTracking (the most popular Win 10 anti-spying script back then) prevented the store app from downloading updates, potentially no longer the case: https://github.com/10se1ucgo/DisableWinTracking

@Atavic

This comment has been minimized.

Copy link

@Atavic Atavic commented Sep 25, 2016

I see as a plus when something like the store doesn't work: it means that the block works.

@FadeMind

This comment has been minimized.

Copy link
Contributor

@FadeMind FadeMind commented Sep 30, 2016

@StevenBlack this issue can be closed cause

590eeac#diff-c36f927ba928cc2158b97e706cc80057R27503

Regards

@Gitoffthelawn

This comment has been minimized.

Copy link
Author

@Gitoffthelawn Gitoffthelawn commented Sep 30, 2016

@FadeMind That's a very useful diff, thank you!

But since it's a static diff, it doesn't provide a continually up-to-date source as the relevant hosts will undoubtedly change over time.

@FadeMind

This comment has been minimized.

Copy link
Contributor

@FadeMind FadeMind commented Sep 30, 2016

@StevenBlack

This comment has been minimized.

Copy link
Owner

@StevenBlack StevenBlack commented Sep 30, 2016

Thanks Tomasz @FadeMind.

@Gitoffthelawn

This comment has been minimized.

Copy link
Author

@Gitoffthelawn Gitoffthelawn commented Oct 1, 2016

@FadeMind Thank you Tomasz!

@monstertruckpa

This comment has been minimized.

Copy link

@monstertruckpa monstertruckpa commented Aug 20, 2017

Hey steven how are u, thanks for the hostslist, but i need in ".txt" mode and direct link, not in a "raw version". please, you could to create in text format for that i can add yours host in "Hosts manager from abelhas". it doesn't not allow to add list in raw without extensions.

@CHEF-KOCH

This comment has been minimized.

Copy link

@CHEF-KOCH CHEF-KOCH commented Aug 20, 2017

Whenever you use the 'raw' format and put the url into (no matter what program or extension) HOSTS manager it always catches the latest version and merges it with your current hosts file.

@monstertruckpa

This comment has been minimized.

Copy link

@monstertruckpa monstertruckpa commented Aug 20, 2017

CHEF-KOCH CANNOT TO DO WHAT YOU SAY.
https://lut.im/vnr0ezcAW4/ClnsNV0e9CCMsYgT.png

@CHEF-KOCH

This comment has been minimized.

Copy link

@CHEF-KOCH CHEF-KOCH commented Aug 20, 2017

Post the url you used over here. HostsMan not really cares about extension at the end, but you can add .txt just on the plain hosts url. But normally it doesn't matter.

For example both are working

https://raw.githubusercontent.com/StevenBlack/hosts/master/data/StevenBlack/hosts
or
https://raw.githubusercontent.com/StevenBlack/hosts/master/data/StevenBlack/hosts.txt

in your case:
https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/social/hosts

or https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/social/hosts.txt
@StevenBlack

This comment has been minimized.

Copy link
Owner

@StevenBlack StevenBlack commented Aug 20, 2017

@monstertruckpa

This comment has been minimized.

Copy link

@monstertruckpa monstertruckpa commented Aug 20, 2017

@StevenBlack The secondary mirror filehost, works perfectly!, gave me not problems. thanks steven, im so happy with abelhas program updater + your yappaplus hosts block list. YUPI!

@CHEF-KOCH

This comment has been minimized.

Copy link

@CHEF-KOCH CHEF-KOCH commented Aug 21, 2017

Remember:

  • Hosts file were never designed to block ads the way people using it today
  • Hosts file is slow compared to solutions which can work with regex
  • It doesn't block all ads on your entire network, only for your local machine
  • Pi-Hole does the job besser and blocks ads on your entire network before they reach your router, so no extensions/root or other stuff necessary
  • DNSname resolution slows down DNS upstream resolution, especially while you use your own DNS with e.g. Pi-Hole, Unbound or in case you need to valide via e.g. DNSCrypt large hosts file are impossible because it causes massive memory hooks and network spikes (depending on which dns mechanism your browser/os uses)
  • Windows 10 by default can bypass certain FDQN's
  • Windows 10 RS 3+ never connects to any of these mentioned hosts, I tried that and I can proof it, depending on the telemetry option you choosed on the setup or afterwards via gpedit.msc it uses akamai CDN's instead (Pro versions calling more often home compared to Enterprise version).
  • Ads can be local pushed cause WUS integrated updates with ads like in the past
  • ASN blocking is 2x faster, more efficient and easier
@ScriptTiger

This comment has been minimized.

Copy link
Contributor

@ScriptTiger ScriptTiger commented Aug 21, 2017

I like your ideas, @CHEF-KOCH. Modern Windows versions come standard with the Windows Firewall which can also block IP ranges/networks. I have a geoIP project I started for local resolutions using the GeoLite2 files from MaxMind, and this could easily be integrated with Steven Black's hosts files by first resolving all the NS records for a domain, and then resolving those IPs to networks/ASNs as you have described above. I see you don't yet support IPv6, I have also been slow to support it just because I have been a bit lazy to implement the calculations for it. But making a seamless process to join Steven Black's hosts files + GeoLite2 + Windows Firewall would be a great smart firewall that doesn't require any additional executables and can be solely scripted to do the resolutions from the hosts files and then configure the firewall. I realize this does nothing for your Linux project, but I just wanted you to know I have definitely been inspired by your comments.

@Atavic

This comment has been minimized.

Copy link

@Atavic Atavic commented Aug 21, 2017

Wait, you are setting rules to Firewall as ADMINISTRATOR.

Windows 10 by default can bypass certain FDQN's

That's because the OS has user profiles with more privileges than Admin (You).

Just use Process Explorer by Sysinternals and look at the properties of some svchost.exe instances. You'll see:

USERDOMAIN: NT AUTHORITY
USERNAME: LOCAL SERVICE

This user profile is the system itself and is greater than Admin.

@Atavic

This comment has been minimized.

Copy link

@Atavic Atavic commented Aug 21, 2017

LOCAL SERVICE can override the Firewall rules set by ADMINISTRATOR

@CHEF-KOCH

This comment has been minimized.

Copy link

@CHEF-KOCH CHEF-KOCH commented Aug 21, 2017

Not correct and neither not an reliable solution:

  • MS patches dnsapi.dll here and there.
  • DNS lookups not use hosts file, before windows 8 they used deprecated gethostbyname() method.
  • User rights have nothing much to do with firewall blocking, cause this is by WMI, which runs at the same rights then the RPC mapper to monitor the events otherwise a normal admin would't even see any of the events, gaining NT autority rights is anyway easy peasy via ms own PsExcec e.g. psexec.exe -i -s %SystemRoot%\system32\cmd.exe but that not changes the fact that windows own solution is related on AD DS, which is a real problem

I'm not going more into details because it's not really helpful since you simply can turn it off and use alternatives (which e.g. also supporting regex and other things to make your life easier).

I'll do a full RS 3 Fiddler/Burp log when the final version is out, to debunk some wrong facts, then we will see if they again selling data or if the only security related meta-data are keept as they promise. Personally I would be comfy as long it's not sold to make money.

@monstertruckpa

This comment has been minimized.

Copy link

@monstertruckpa monstertruckpa commented Aug 21, 2017

Thank You CHEFKOCH for your explanation. very helpful all what you tell us.

@ScriptTiger

This comment has been minimized.

Copy link
Contributor

@ScriptTiger ScriptTiger commented Aug 22, 2017

@Tobias-B-Besemer, maybe you're crew might be interested in another project? A binary package to configure both the hosts file and Windows Firewall (see my post above). I have been dabbling in my geoIP project for a while just because I do have a day job and haven't gotten to completing the actual search script yet, but the core calculations for IPv4 are all there if you want to use my script as pseudo code for your project. Or you can wait for a while for me to draft up a script which actually does all that and then you can use it as pseudo code and port it to C#.

@Tobias-B-Besemer

This comment has been minimized.

Copy link
Contributor

@Tobias-B-Besemer Tobias-B-Besemer commented Aug 22, 2017

CC @D4rkCr0w, as he make the code...

(For the others, we talk about: https://github.com/LV-Crew/HostsManager/ )
(Issue-Reports are welcome!)

@maravento

This comment has been minimized.

Copy link

@maravento maravento commented Apr 25, 2019

Hi Steve. I need to exclude telemetry, from whiteurls. Do you have any file (download url) about this? Thanks in advance

@StevenBlack

This comment has been minimized.

Copy link
Owner

@StevenBlack StevenBlack commented Apr 25, 2019

I'm not sure I understand, @maravento.

The whiteurls list will need modification to prefix 0.0.0.0 or 127.0.0.1 on each line, then you'll need to add its details to an update.json file, then generate your own hosts file using updatehosts.py.

Is that what you're asking?

@maravento

This comment has been minimized.

Copy link

@maravento maravento commented Apr 25, 2019

I just need a telemetry list. It does not matter if the list has prefixes or not (0.etc or 127.etc)

@ScriptTiger

This comment has been minimized.

Copy link
Contributor

@ScriptTiger ScriptTiger commented Apr 25, 2019

There is no separate extension or specific data source for telemetry, they are all grouped together with malware, adware, and other generic unwanted players. You would have to go through and manually read the comments to separate out telemetry from everything else. Our sources are usually pretty well commented, so that is probably your best option.

@maravento

This comment has been minimized.

Copy link

@maravento maravento commented Apr 25, 2019

It would be good to build a list dedicated exclusively to telemetry (or a project). I am trying to make one (HERE), which I have obtained from different sources, but I have not been able to use it because it has many false positives. Keep investigating. Thanks for your help

@ScriptTiger

This comment has been minimized.

Copy link
Contributor

@ScriptTiger ScriptTiger commented Apr 25, 2019

If someone wants to take on a personal project, you could reach out to each of the data sources to see if they would be willing to use standardized tagging of some kind using comments to categorize each entry for why they curate each entry. If all of the data sources can agree on a standard, then the aggregate here can easily be filtered through a script to separate out categories further. There's nothing really we can do downstream once it gets to our end though and all of the data sources use their own disparate curation methods

@maravento

This comment has been minimized.

Copy link

@maravento maravento commented Apr 25, 2019

here is my test repository, in case you want to continue with this project (since I do not work with windows host format) https://github.com/maravento/telemetry
pd: the repository will be removed soon

@Tobias-B-Besemer

This comment has been minimized.

Copy link
Contributor

@Tobias-B-Besemer Tobias-B-Besemer commented Apr 26, 2019

There exist a PowerShell project https://github.com/W4RH4WK/Debloat-Windows-10 but there seems to be a lot of issues with it (microsoft/MS-DOS#395 (comment)).

@rautamiekka

This comment has been minimized.

Copy link

@rautamiekka rautamiekka commented Apr 26, 2019

There exist a PowerShell project https://github.com/W4RH4WK/Debloat-Windows-10 but there seems to be a lot of issues with it (Microsoft/MS-DOS#395 (comment)).

Don't use the one by W4RH4WK ! It's buggy beyond functional, use https://github.com/Sycnex/Windows10Debloater instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.