Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

iptables reject rule doesn't work if FORWARD rules (to DOCKER) present and take precedence over INPUT #2376

Closed
aario opened this Issue Mar 21, 2019 · 6 comments

Comments

Projects
None yet
2 participants
@aario
Copy link

aario commented Mar 21, 2019

Environment:

  • Fail2Ban version (including any possible distribution suffixes):
    0.9.3-a
  • OS, including release name/version:
NAME="Ubuntu"
VERSION="16.04.6 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.6 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial
  • Fail2Ban installed via OS/distribution mechanisms
  • You have not applied any additional foreign patches to the codebase
  • Some customizations were done to the configuration (provide details below is so)

The issue:

fail2ban simply doesn't work.

sudo fail2ban-client status nginx-http-auth
Status for the jail: nginx-http-auth
|- Filter
|  |- Currently failed: 1
|  |- Total failed:     8
|  `- File list:        /var/log/nginx/error.log
`- Actions
   |- Currently banned: 1
   |- Total banned:     1
   `- Banned IP list:   99.99.99.99

Then in iptables I have hostname instead of the ip:

$ sudo iptables -L | grep REJECT
REJECT     all  --  the-ips-hostname.tld  anywhere             reject-with icmp-port-unreachable

And in log I can see the fail2ban banned the ip after multiple failures:

2019-03-21 14:37:29,868 fail2ban.filter         [29227]: INFO    [nginx-http-auth] Found 99.99.99.99
2019-03-21 14:37:30,457 fail2ban.filter         [29227]: INFO    [nginx-http-auth] Found 99.99.99.99
2019-03-21 14:37:31,091 fail2ban.filter         [29227]: INFO    [nginx-http-auth] Found 99.99.99.99
2019-03-21 14:37:31,648 fail2ban.filter         [29227]: INFO    [nginx-http-auth] Found 99.99.99.99
2019-03-21 14:37:32,235 fail2ban.filter         [29227]: INFO    [nginx-http-auth] Found 99.99.99.99
2019-03-21 14:37:32,575 fail2ban.actions        [29227]: NOTICE  [nginx-http-auth] Ban 99.99.99.99

And nothing happens. The ip 99.99.99.99 can still connect and try all possible passwords.
I also tried this in config:

/etc/fail2ban/jail.conf:usedns = no

Still the same. Nothing has changed.

Steps to reproduce

All default nginx and fail2ban setup. Enable nginx-http-auth jail and see that iptables rule is just ineffective possible because being banned by hostname instead of ip.

Expected behavior

fail2ban should work.

Observed behavior

The ip 99.99.99.99 can still connect and try all possible passwords on nginx.

Any additional information

Configuration, dump and another helpful excerpts

Any customizations done to /etc/fail2ban/ configuration

/etc/fail2ban/jail.conf:usedns = no

And:

/etc/fail2ban/jail.d/defaults-debian.conf-[nginx-http-auth]
/etc/fail2ban/jail.d/defaults-debian.conf:enabled = true

Relevant parts of /var/log/fail2ban.log file:

preferably obtained while running fail2ban with loglevel = 4

2019-03-21 14:37:29,868 fail2ban.filter         [29227]: INFO    [nginx-http-auth] Found 99.99.99.99
2019-03-21 14:37:30,457 fail2ban.filter         [29227]: INFO    [nginx-http-auth] Found 99.99.99.99
2019-03-21 14:37:31,091 fail2ban.filter         [29227]: INFO    [nginx-http-auth] Found 99.99.99.99
2019-03-21 14:37:31,648 fail2ban.filter         [29227]: INFO    [nginx-http-auth] Found 99.99.99.99
2019-03-21 14:37:32,235 fail2ban.filter         [29227]: INFO    [nginx-http-auth] Found 99.99.99.99
2019-03-21 14:37:32,575 fail2ban.actions        [29227]: NOTICE  [nginx-http-auth] Ban 99.99.99.99

Relevant lines from monitored log files in question:

2019/03/21 14:37:30 [error] 6#6: *39 user "username": password mismatch, client: 99.99.99.99, server: myserver.tld, request: "GET /index.php/ HTTP/1.1", host: "myserver.tld"
2019/03/21 14:37:31 [error] 6#6: *39 user "username": password mismatch, client: 99.99.99.99, server: myserver.tld, request: "GET /index.php/ HTTP/1.1", host: "myserver.tld"
2019/03/21 14:37:31 [error] 6#6: *39 user "username": password mismatch, client: 99.99.99.99, server: myserver.tld, request: "GET /index.php/ HTTP/1.1", host: "myserver.tld"
2019/03/21 14:37:32 [error] 6#6: *39 user "username": password mismatch, client: 99.99.99.99, server: myserver.tld, request: "GET /index.php/ HTTP/1.1", host: "myserver.tld"
2019/03/21 14:37:32 [error] 6#6: *39 user "username": password mismatch, client: 99.99.99.99, server: myserver.tld, request: "GET /index.php/ HTTP/1.1", host: "myserver.tld"
2019/03/21 14:37:33 [error] 6#6: *39 user "username": password mismatch, client: 99.99.99.99, server: myserver.tld, request: "GET /index.php/ HTTP/1.1", host: "myserver.tld"
2019/03/21 14:37:33 [error] 6#6: *39 user "username": password mismatch, client: 99.99.99.99, server: myserver.tld, request: "GET /index.php/ HTTP/1.1", host: "myserver.tld"

@aario aario changed the title iptables rejct rule created by hostname and it doesn't work iptables reject rule created by hostname and it doesn't work Mar 21, 2019

@aario

This comment has been minimized.

Copy link
Author

aario commented Mar 21, 2019

I also tried manually ban the ip using:

sudo iptables -I f2b-nginx-http-auth 1 -s 99.99.99.99 -j REJECT --reject-with icmp-port-unreachable

And:

sudo iptables -I f2b-nginx-http-auth 1 -s 99.99.99.99 -j DROP

Therefore my iptables became like this:

DROP       all  --  hostname.tld  anywhere            
REJECT     all  --  hostname.tld  anywhere             reject-with icmp-port-unreachable
REJECT     all  --  hostname.tld  anywhere             reject-with icmp-port-unreachable

First and second created by above commands and third one by fail2ban. All rules are inserted by hostname instead of ip! Also, the ip (which is my ip actually) is still not banned!

@aario

This comment has been minimized.

Copy link
Author

aario commented Mar 21, 2019

So some new clue. If we add the reject rule to INPUT chain it works and it blocks. If we add it to fail2ban chain:

sudo iptables -I f2b-nginx-http-auth 1 -s 99.99.99.99 -j DROP

It doesn’t.

@sebres

This comment has been minimized.

Copy link
Member

sebres commented Mar 22, 2019

Then in iptables I have hostname instead of the ip:

Wrong, it is the resolver inside of listing in iptables. Use iptables -nL instead of iptables -L.

I also tried this in config ... usedns = no

The settings of option usedns = no is not really necessary here (if you have IPs only), but does not disturb (so if host names are not expected in log, it is simply faster and more precise).

If we add the reject rule to INPUT chain it works and it blocks. If we add it to fail2ban chain... It doesn’t.

I don't see how the chain is created, please provide also the output of INPUT chain (where f2b-nginx-http-auth would be basically referenced)...
E. g. which ports are configured for jail (chain)?

@aario

This comment has been minimized.

Copy link
Author

aario commented Mar 22, 2019

Ok. Let's take a look at the whole iptables then:

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
f2b-nginx-http-auth  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 80,443
f2b-sshd   tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 22
f2b-nginx-http-auth  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 80,443
f2b-sshd   tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 22
ufw-before-logging-input  all  --  0.0.0.0/0            0.0.0.0/0           
ufw-before-input  all  --  0.0.0.0/0            0.0.0.0/0           
ufw-after-input  all  --  0.0.0.0/0            0.0.0.0/0           
ufw-after-logging-input  all  --  0.0.0.0/0            0.0.0.0/0           
ufw-reject-input  all  --  0.0.0.0/0            0.0.0.0/0           
ufw-track-input  all  --  0.0.0.0/0            0.0.0.0/0           
DROP       all  --  XX.XX.XX.XX         0.0.0.0/0           #THIS IS THE IP I ADDED MANUALLY AND IT WORKS

Chain FORWARD (policy DROP)
target     prot opt source               destination         
DOCKER-USER  all  --  0.0.0.0/0            0.0.0.0/0           
DOCKER-ISOLATION-STAGE-1  all  --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
DOCKER     all  --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
DOCKER     all  --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
DOCKER     all  --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
DOCKER     all  --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           
ufw-before-logging-forward  all  --  0.0.0.0/0            0.0.0.0/0           
ufw-before-forward  all  --  0.0.0.0/0            0.0.0.0/0           
ufw-after-forward  all  --  0.0.0.0/0            0.0.0.0/0           
ufw-after-logging-forward  all  --  0.0.0.0/0            0.0.0.0/0           
ufw-reject-forward  all  --  0.0.0.0/0            0.0.0.0/0           
ufw-track-forward  all  --  0.0.0.0/0            0.0.0.0/0           

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
ufw-before-logging-output  all  --  0.0.0.0/0            0.0.0.0/0           
ufw-before-output  all  --  0.0.0.0/0            0.0.0.0/0           
ufw-after-output  all  --  0.0.0.0/0            0.0.0.0/0           
ufw-after-logging-output  all  --  0.0.0.0/0            0.0.0.0/0           
ufw-reject-output  all  --  0.0.0.0/0            0.0.0.0/0           
ufw-track-output  all  --  0.0.0.0/0            0.0.0.0/0           

Chain DOCKER (4 references)
target     prot opt source               destination         
ACCEPT     tcp  --  0.0.0.0/0            ZZZ.ZZ.ZZ.ZZ           tcp dpt:80
ACCEPT     tcp  --  0.0.0.0/0            WWW.WW.WW.WW           tcp dpt:3306

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target     prot opt source               destination         
DOCKER-ISOLATION-STAGE-2  all  --  0.0.0.0/0            0.0.0.0/0           
DOCKER-ISOLATION-STAGE-2  all  --  0.0.0.0/0            0.0.0.0/0           
DOCKER-ISOLATION-STAGE-2  all  --  0.0.0.0/0            0.0.0.0/0           
DOCKER-ISOLATION-STAGE-2  all  --  0.0.0.0/0            0.0.0.0/0           
RETURN     all  --  0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-ISOLATION-STAGE-2 (4 references)
target     prot opt source               destination         
DROP       all  --  0.0.0.0/0            0.0.0.0/0           
DROP       all  --  0.0.0.0/0            0.0.0.0/0           
DROP       all  --  0.0.0.0/0            0.0.0.0/0           
DROP       all  --  0.0.0.0/0            0.0.0.0/0           
RETURN     all  --  0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-USER (1 references)
target     prot opt source               destination         
RETURN     all  --  0.0.0.0/0            0.0.0.0/0           

Chain f2b-nginx-http-auth (2 references)
target     prot opt source               destination         
#THIS IS THE IP FAIL2BAN ADDED AUTOMATICALLY AND IT DOESN'T WORK
REJECT     all  --  YY.YY.YY.YY        0.0.0.0/0            reject-with icmp-port-unreachable
RETURN     all  --  0.0.0.0/0            0.0.0.0/0           
RETURN     all  --  0.0.0.0/0            0.0.0.0/0           

Chain f2b-sshd (2 references)
target     prot opt source               destination         
RETURN     all  --  0.0.0.0/0            0.0.0.0/0           
RETURN     all  --  0.0.0.0/0            0.0.0.0/0           

Chain ufw-after-forward (1 references)
target     prot opt source               destination         

Chain ufw-after-input (1 references)
target     prot opt source               destination         

Chain ufw-after-logging-forward (1 references)
target     prot opt source               destination         

Chain ufw-after-logging-input (1 references)
target     prot opt source               destination         

Chain ufw-after-logging-output (1 references)
target     prot opt source               destination         

Chain ufw-after-output (1 references)
target     prot opt source               destination         

Chain ufw-before-forward (1 references)
target     prot opt source               destination         

Chain ufw-before-input (1 references)
target     prot opt source               destination         

Chain ufw-before-logging-forward (1 references)
target     prot opt source               destination         

Chain ufw-before-logging-input (1 references)
target     prot opt source               destination         

Chain ufw-before-logging-output (1 references)
target     prot opt source               destination         

Chain ufw-before-output (1 references)
target     prot opt source               destination         

Chain ufw-reject-forward (1 references)
target     prot opt source               destination         

Chain ufw-reject-input (1 references)
target     prot opt source               destination         

Chain ufw-reject-output (1 references)
target     prot opt source               destination         

Chain ufw-track-forward (1 references)
target     prot opt source               destination         

Chain ufw-track-input (1 references)
target     prot opt source               destination         

Chain ufw-track-output (1 references)
target     prot opt source               destination         
@aario

This comment has been minimized.

Copy link
Author

aario commented Mar 22, 2019

So, this command fixed the problem:

sudo iptables -I FORWARD -j f2b-nginx-http-auth

How?
I had also a chain created by DOCKER which contained rules contradicting with those imposed by fail2ban. And IP tables was prioritizing FORWARD chain over INPUT chain where fail2ban puts its own chain in. So inserting fail2ban chain into FORWARD chain prioritized fail2ban above docker and now the whole thing works.

@sebres

This comment has been minimized.

Copy link
Member

sebres commented Mar 22, 2019

Ok (tried to write a comment, but xed).

Also it is yours (3rd party) issue. :)

Glad it works now. Thus closed.

@sebres sebres closed this Mar 22, 2019

@sebres sebres changed the title iptables reject rule created by hostname and it doesn't work iptables reject rule doesn't work if FORWARD rules (to DOCKER) present and takes precedence over INPUT Mar 22, 2019

@sebres sebres changed the title iptables reject rule doesn't work if FORWARD rules (to DOCKER) present and takes precedence over INPUT iptables reject rule doesn't work if FORWARD rules (to DOCKER) present and take precedence over INPUT Mar 22, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.