Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new feature added: sync down global blacklists #874

Open
klepsydra opened this issue Dec 1, 2014 · 5 comments
Open

new feature added: sync down global blacklists #874

klepsydra opened this issue Dec 1, 2014 · 5 comments

Comments

@klepsydra
Copy link

Hello, I have a small contribution to share, but I do not know where is the best place to post it, nor if/where to submit into the source repository https://github.com/fail2ban/fail2ban

After thoroughly searching but not finding an existing way to keep my local fail2ban updating with blacklists of known attackers, similar to the SERVER_SYNC option Denyhosts offers, I described what I would code up at https://plus.google.com/102676720348185803844/posts/XoDRGQU4i4A

Now, I have a short script to be run under some cron schedule, at /etc/cron.daily/sync.fail2ban.sh for now. Here are its contents: https://gist.github.com/klepsydra/ecf975984b32b1c8291a

For now, it only takes these two lists from https://www.blocklist.de/en/export.html of attackers that had already been reported by others around the world:

Typ: ssh
Description: All IP addresses which have been reported within the last 48 hours as having run attacks on the service SSH.

Typ: bruteforcelogin
Description: All IPs which attacks Joomlas, Wordpress and other Web-Logins with Brute-Force

Combines them, removes duplicates, etc. and blocks them in your own local iptables, just like fail2ban would as if they had already attacked you over ssh (tried to exploit vulnerabilities).

Every day it checks for changes, and doesn't forget to unlist those items which are no longer listed as a threat.

I've been testing for a few days and making adjustments, like filtering out IPv6 addresses, since I didn't work out how to handle them in iptables (though that, and many other features seem simple).
Works pretty well for my servers though!

Please feel free to improve and generalize.
Marcos

@klepsydra
Copy link
Author

Interesting to note:

Every time I had gotten a notification from fail2ban similar to:
(only one example of maybe 50-60 in the logs & emails DAILY, mostly from China)

[Fail2Ban] ssh: banned 122.225.109.125 from my server.com
...
The IP 122.225.109.125 has just been banned by Fail2Ban after
6 attempts against ssh.
...
Here are more information about 122.225.109.125:
% [whois.apnic.net]
irt: IRT-CHINANET-ZJ
address: Hangzhou, 288 fucun Road, China
...
Lines containing IP:122.225.109.125 in /var/log/auth.log

Dec 1 11:07:57 vm-stage sshd[12076]: Failed password for root from 122.225.109.125 port 21417 ssh2
Dec 1 11:08:13 vm-stage sshd[12076]: message repeated 5 times: [ Failed password for root from 122.225.109.125 port 21417 ssh2]

Then proceeded to manually research that IP at portals such as:
https://www.blocklist.de/en/view.html?ip=122.225.109.125
http://www.abuseipdb.com/report-history/122.225.109.125

It turns out that 122.225.109.125 had already been a long-standing known attacker, already blacklisted on public sites including blocklist.de which I now sync with, even months ago!

Who Date Comment Categories
www.blocklist.de Sep 28, 2014 [Sun Sep 28 16:29:24 2014] Failed password for r.r from 122.225.109.125 port 38509 ssh2
[Sun Sep 28 16:29:24 2014] Failed password for r.r from 122.225.109.125 port 38519 ssh2
[Sun Sep 28 16:29:25 2014] Failed password for r.r from 122.225.109.125 port 38470 ssh2
[Sun Sep 28 16:29:25 2014] Failed password for r.r from 122.225.109.125 port 38778 ssh2

........

http://www.blocklist.de/en/view.html?ip=122.225.109.125 Hacking, FTP Brute-force

If that isn't compelling evidence that such a feature for fail2ban was missing, then I don't know what is.

@sebres
Copy link
Contributor

sebres commented Dec 1, 2014

small remark for me for #716:

  • before increasing a ban time, observer can check ip is bad within configured blacklist's, to increase a time more aggressive up to permanent.

@leto
Copy link

leto commented Dec 29, 2014

This looks like a useful feature that I would use. Thanks! Some notes: Your script downloads files over HTTP and executes parts of them as root. That greatly concerns me. I suggest verifying the MD5s that blocklist.de provides and using HTTPS. For instance, https://lists.blocklist.de/lists/all.txt works (but has a self-signed cert). I would also verify chunks of data look reasonable before sending them commands that execute as root.

@klepsydra
Copy link
Author

@leto It does download data files (how else?) from canonical sites, but doesn't execute anything in them. In fact, there are additional filters that try to weed out anything but lines containing IPv4 addresses, like comments, IPv6 addresses, or just garbage. Underneath, iptables will simply error out and quit upon a source address it doesn't understand. Harmless.

Only root access can alter the iptables ultimately, regardless of where knowledge of what to block came from. This mechanism is how Denyhosts works too; Fail2ban was merely lacking it.

@omaidb
Copy link

omaidb commented Sep 5, 2023

这个脚本看起来很棒

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants