Uncompressed option?

[pallebone]

Is there an uncompressed version of the blocklists anywhere (the link is tar.gz on the page). Reason I ask is pihole does not support compressed formats as a blocklist.

Kind regards
Peter

[T145]

Welcome Peter!

There used to be such an option, but I've decided against it for a few reasons.

1. If I commit large files directly to the repository, the repo's history grows exponentially, making it tedious for people to clone and contribute on slower connections. This is why I've opted to have releases, unlike other repositories. I'm also aiming to respect GitHub's resources and resource consumption.
2. This list will get bigger before it gets smaller, and as it grows figuring out distribution becomes more complicated. This is why I've opted to just release archives of each type instead of specializing them to a specific platform. The specialization is rather trivial for an end-user and just gets more complicated for me.
3. You can easily set up a bash script to download the archives w/ your choice utility and extract the archive: https://linuxize.com/post/how-to-create-and-extract-archives-using-the-tar-command-in-linux/#extracting-tar-archive
   The links you see in the README release table are permanent, so you won't need to update them.

[T145]

@pallebone

One thing I can do is make a build artifact that's a text file with all my sources on one line. This is a feature I didn't know about previously: https://discourse.pi-hole.net/t/how-to-add-blocklists-v5-and-later/32127
However, if I'm understanding correctly, finding duplicates may be a pain. It also hasn't been personally tested. Would this be something you're interested in?

[pallebone]

Hi,

Yes please this would be very helpful. I can check each list and see if they are all in hosts format and use the ones that are. Obviously as you say it would not be as efficient as yours without duplicates and if some are in other formats would not be possible to use but as a workaround would help especially if most work its better than nothing :)

Pete

[pallebone]

Also not sure how you get the ipv4 blocklist but I also actually maintain one at https://github.com/pallebone/StrictBlockPAllebone if that srt of thing is of any interest to you and you wanted to take a look to see if it was of any interest. Sorry if thats not of interest just thought I would mention it in case :)

Pete

[T145]

Adding the Spamhaus sources is on my TODO list since tracking them down required working around their paywall. Once those are in I suppose you can just use this project over yours and have it be updated daily. I'll add an issue detailing the build artifact and reference this issue.

[T145]

@pallebone Enjoy your text release 😉

[pallebone]

Ah thanks. Am I using this link here?

https://github.com/T145/the-blacklist/releases/latest/download/black_domain.txt

[T145]

Yes.

[pallebone]

Just loaded it now, the list stats are:
[i] Target: https://github.com/T145/the-blacklist/releases/latest/download/black_domain.txt
[✓] Status: Retrieval successful
[i] Analyzed 9027498 domains