One more time, sshd not filtered

[metal3d]

Environment:

• Fail2Ban version (including any possible distribution suffixes): 0.10.1 + filters taken from 0.10.5 version
• OS, including release name/version: Debian 10.3
• Fail2Ban installed via OS/distribution mechanisms
• You have not applied any additional foreign patches to the codebase
• Some customizations were done to the configuration (provide details below is so)

I took the entire filter files from 0.10.5

I added this:

$ cat /etc/fail2ban/jail.d/ssh.conf 
[sshd]
port = ssh,sftp
maxretry = 3
enabled = true
findtime = 3600
bantime = 86400
logpath = %(sshd_log)s
backend = %(sshd_backend)s

The issue:

My server has got a lot of ssh connection attempts to be filtered and banned, but nothing happens.

Steps to reproduce

See this output:

$ fail2ban-regex /var/log/auth.log /etc/fail2ban/filter.d/sshd.conf
Lines: 379224 lines, 0 ignored, 0 matched, 379224 missed
[processed in 16.13 sec]

Missed line(s): too many to print.  Use --print-all-missed to print all 379224 lines

But as you can see, there are several lines that should match:

Apr 18 09:09:41 teamate sshd[26702]: Failed password for root from 49.88.112.73 port 30724 ssh2
Apr 18 09:09:42 teamate sshd[26724]: Failed password for root from 51.15.136.91 port 57280 ssh2
Apr 18 09:09:42 teamate sshd[26724]: Received disconnect from 51.15.136.91 port 57280:11: Bye Bye [preauth]
Apr 18 09:09:42 teamate sshd[26724]: Disconnected from authenticating user root 51.15.136.91 port 57280 [preauth]
Apr 18 09:09:44 teamate sshd[26702]: Failed password for root from 49.88.112.73 port 30724 ssh2
Apr 18 09:09:47 teamate sshd[26702]: Failed password for root from 49.88.112.73 port 30724 ssh2
Apr 18 09:09:47 teamate sshd[26702]: Received disconnect from 49.88.112.73 port 30724:11:  [preauth]
Apr 18 09:09:47 teamate sshd[26702]: Disconnected from authenticating user root 49.88.112.73 port 30724 [preauth]
Apr 18 09:09:47 teamate sshd[26702]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=49.88.112.73  user=root
Apr 18 09:10:29 teamate sshd[26975]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=80.211.22.40  user=root
Apr 18 09:10:29 teamate sshd[26961]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=49.88.112.73  user=root
Apr 18 09:10:32 teamate sshd[26975]: Failed password for root from 80.211.22.40 port 35314 ssh2
Apr 18 09:10:32 teamate sshd[26961]: Failed password for root from 49.88.112.73 port 41407 ssh2

Thanks a lot

[youtous]

Hi,
I was not able to reproduce your issue :

Click to expand

╰─λ python2.7 ./bin/fail2ban-regex var.log ./config/filter.d/sshd.conf -v                                                                                                   1 < 11:25:46
Fail2ban 0.10.6-dev test suite. Python 2.7.17 (default, Apr  8 2020, 16:59:37) [GCC 9.3.0]. Please wait...

Running tests
=============

Use   failregex filter file : sshd, basedir: ./config
Use         maxlines : 1
Use      datepattern : Default Detectors
Use         log file : var.log
Use         encoding : UTF-8


Results
=======

Failregex: 12 total
|-  #) [# of hits] regular expression
|   1) [0] ^[aA]uthentication (?:failure|error|failed) for <F-USER>.*</F-USER> from <HOST>( via \S+)?(?: (?:port \d+|on \S+|\[preauth\])){0,3}\s*$
|   2) [0] ^User not known to the underlying authentication module for <F-USER>.*</F-USER> from <HOST>(?: (?:port \d+|on \S+|\[preauth\])){0,3}\s*$
|   3) [0] ^Failed publickey for invalid user <F-USER>(?P<cond_user>\S+)|(?:(?! from ).)*?</F-USER> from <HOST>(?: (?:port \d+|on \S+)){0,2}(?: ssh\d*)?(?(cond_user): |(?:(?:(?! from ).)*)$)
|   4) [6] ^Failed (?:<F-NOFAIL>publickey</F-NOFAIL>|\S+) for (?P<cond_inv>invalid user )?<F-USER>(?P<cond_user>\S+)|(?(cond_inv)(?:(?! from ).)*?|[^:]+)</F-USER> from <HOST>(?: (?:port \d+|on \S+)){0,2}(?: ssh\d*)?(?(cond_user): |(?:(?:(?! from ).)*)$)
|      49.88.112.73  Sat Apr 18 09:09:41 2020
|      51.15.136.91  Sat Apr 18 09:09:42 2020
|      49.88.112.73  Sat Apr 18 09:09:44 2020
|      49.88.112.73  Sat Apr 18 09:09:47 2020
|      80.211.22.40  Sat Apr 18 09:10:32 2020
|      49.88.112.73  Sat Apr 18 09:10:32 2020
|   5) [0] ^<F-USER>ROOT</F-USER> LOGIN REFUSED FROM <HOST>
|   6) [0] ^[iI](?:llegal|nvalid) user <F-USER>.*?</F-USER> from <HOST>(?: (?:port \d+|on \S+|\[preauth\])){0,3}\s*$
|   7) [0] ^User <F-USER>.+</F-USER> from <HOST> not allowed because not listed in AllowUsers(?: (?:port \d+|on \S+|\[preauth\])){0,3}\s*$
|   8) [0] ^User <F-USER>.+</F-USER> from <HOST> not allowed because listed in DenyUsers(?: (?:port \d+|on \S+|\[preauth\])){0,3}\s*$
|   9) [0] ^User <F-USER>.+</F-USER> from <HOST> not allowed because not in any group(?: (?:port \d+|on \S+|\[preauth\])){0,3}\s*$
|  10) [0] ^refused connect from \S+ \(<HOST>\)
|  11) [0] ^Received <F-MLFFORGET>disconnect</F-MLFFORGET> from <HOST>(?: (?:port \d+|on \S+)){0,2}:\s*3: .*: Auth fail(?: (?:port \d+|on \S+|\[preauth\])){0,3}\s*$
|  12) [0] ^User <F-USER>.+</F-USER> from <HOST> not allowed because a group is listed in DenyGroups(?: (?:port \d+|on \S+|\[preauth\])){0,3}\s*$
|  13) [0] ^User <F-USER>.+</F-USER> from <HOST> not allowed because none of user's groups are listed in AllowGroups(?: (?:port \d+|on \S+|\[preauth\])){0,3}\s*$
|  14) [2] ^<F-NOFAIL>pam_[a-z]+\(sshd:auth\):\s+authentication failure;</F-NOFAIL>(?:\s+(?:(?:logname|e?uid|tty)=\S*)){0,4}\s+ruser=<F-ALT_USER>\S*</F-ALT_USER>\s+rhost=<HOST>(?:\s+user=<F-USER>\S*</F-USER>)?(?: (?:port \d+|on \S+|\[preauth\])){0,3}\s*$
|      80.211.22.40  Sat Apr 18 09:10:29 2020
|      49.88.112.73  Sat Apr 18 09:10:29 2020
|  15) [0] ^maximum authentication attempts exceeded for <F-USER>.*</F-USER> from <HOST>(?: (?:port \d+|on \S+)){0,2}(?: ssh\d*)?(?: (?:port \d+|on \S+|\[preauth\])){0,3}\s*$
|  16) [0] ^User <F-USER>.+</F-USER> not allowed because account is locked(?: (?:port \d+|on \S+|\[preauth\])){0,3}\s*
|  17) [0] ^<F-MLFFORGET>Disconnecting</F-MLFFORGET>(?: from)?(?: (?:invalid|authenticating)) user <F-USER>\S+</F-USER> <HOST>(?: (?:port \d+|on \S+)){0,2}:\s*Change of username or service not allowed:\s*.*\[preauth\]\s*$
|  18) [0] ^Disconnecting: Too many authentication failures(?: for <F-USER>.+?</F-USER>)?(?: (?:port \d+|on \S+|\[preauth\])){0,3}\s*$
|  19) [2] ^<F-NOFAIL>Received <F-MLFFORGET>disconnect</F-MLFFORGET></F-NOFAIL> from <HOST>(?: (?:port \d+|on \S+)){0,2}:\s*11:
|      51.15.136.91  Sat Apr 18 09:09:42 2020
|      49.88.112.73  Sat Apr 18 09:09:47 2020
|  20) [2] ^<F-NOFAIL><F-MLFFORGET>(Connection closed|Disconnected)</F-MLFFORGET></F-NOFAIL> (?:by|from)(?: (?:invalid|authenticating) user <F-USER>\S+|.+?</F-USER>)? <HOST>(?:(?: (?:port \d+|on \S+|\[preauth\])){0,3}\s*|\s*)$
|      51.15.136.91  Sat Apr 18 09:09:42 2020
|      49.88.112.73  Sat Apr 18 09:09:47 2020
|  21) [0] ^<F-MLFFORGET><F-MLFGAINED>Accepted \w+</F-MLFGAINED></F-MLFFORGET> for <F-USER>\S+</F-USER> from <HOST>(?:\s|$)
|  22) [0] ^<F-NOFAIL>Connection from</F-NOFAIL> <HOST>
`-

Ignoreregex: 0 total

Date template hits:
|- [# of hits] date format
|  [13] {^LN-BEG}(?:DAY )?MON Day %k:Minute:Second(?:\.Microseconds)?(?: ExYear)?
|  [0] {^LN-BEG}ExYear(?P<_sep>[-/.])Month(?P=_sep)Day(?:T|  ?)24hour:Minute:Second(?:[.,]Microseconds)?(?:\s*Zone offset)?
|  [0] {^LN-BEG}(?:DAY )?MON Day ExYear %k:Minute:Second(?:\.Microseconds)?
|  [0] {^LN-BEG}Day(?P<_sep>[-/])Month(?P=_sep)(?:ExYear|ExYear2) %k:Minute:Second
|  [0] {^LN-BEG}Day(?P<_sep>[-/])MON(?P=_sep)ExYear[ :]?24hour:Minute:Second(?:\.Microseconds)?(?: Zone offset)?
|  [0] {^LN-BEG}Month/Day/ExYear:24hour:Minute:Second
|  [0] {^LN-BEG}Month-Day-ExYear %k:Minute:Second(?:\.Microseconds)?
|  [0] {^LN-BEG}Epoch
|  [0] {^LN-BEG}ExYear2ExMonthExDay  ?24hour:Minute:Second
|  [0] {^LN-BEG}MON Day, ExYear 12hour:Minute:Second AMPM
|  [0] {^LN-BEG}ExYearExMonthExDay(?:T|  ?)Ex24hourExMinuteExSecond(?:[.,]Microseconds)?(?:\s*Zone offset)?
|  [0] {^LN-BEG}(?:Zone name )?(?:DAY )?MON Day %k:Minute:Second(?:\.Microseconds)?(?: ExYear)?
|  [0] {^LN-BEG}(?:Zone offset )?(?:DAY )?MON Day %k:Minute:Second(?:\.Microseconds)?(?: ExYear)?
|  [0] {^LN-BEG}TAI64N
|  [0] {^LN-BEG}24hour:Minute:Second
|  [0] ^<Month/Day/ExYear2@24hour:Minute:Second>
|  [0] ^MON-Day-ExYear2 %k:Minute:Second
`-

Lines: 13 lines, 6 ignored, 6 matched, 1 missed
[processed in 0.00 sec]

|- Ignored line(s):
|  Apr 18 09:09:42 teamate sshd[26724]: Received disconnect from 51.15.136.91 port 57280:11: Bye Bye [preauth]
|  Apr 18 09:09:42 teamate sshd[26724]: Disconnected from authenticating user root 51.15.136.91 port 57280 [preauth]
|  Apr 18 09:09:47 teamate sshd[26702]: Received disconnect from 49.88.112.73 port 30724:11:  [preauth]
|  Apr 18 09:09:47 teamate sshd[26702]: Disconnected from authenticating user root 49.88.112.73 port 30724 [preauth]
|  Apr 18 09:10:29 teamate sshd[26975]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=80.211.22.40  user=root
|  Apr 18 09:10:29 teamate sshd[26961]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=49.88.112.73  user=root
`-
|- Missed line(s):
|  Apr 18 09:09:47 teamate sshd[26702]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=49.88.112.73  user=root
`-

What is the content of your /etc/fail2ban/filter.d/sshd.conf ?
Can you run the fail2ban-regex using -v argument?
Thank you

[sebres]

Latest version of sshd filter (>= 0.10.5) tested with your excerpt finds 8 failures:

$ fail2ban-regex -o msg /tmp/gh-2695.txt sshd
Apr 18 09:09:41 teamate sshd[26702]: Failed password for root from 49.88.112.73 port 30724 ssh2
Apr 18 09:09:42 teamate sshd[26724]: Failed password for root from 51.15.136.91 port 57280 ssh2
Apr 18 09:09:44 teamate sshd[26702]: Failed password for root from 49.88.112.73 port 30724 ssh2
Apr 18 09:09:47 teamate sshd[26702]: Failed password for root from 49.88.112.73 port 30724 ssh2
Apr 18 09:10:29 teamate sshd[26975]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=80.211.22.40  user=root
Apr 18 09:10:32 teamate sshd[26975]: Failed password for root from 80.211.22.40 port 35314 ssh2
Apr 18 09:10:29 teamate sshd[26961]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=49.88.112.73  user=root
Apr 18 09:10:32 teamate sshd[26961]: Failed password for root from 49.88.112.73 port 41407 ssh2

I guess it is outdated or contains some bug that was fixed hereafter.