New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

no 'host' group error (default filters) #1723

Closed
microcreators opened this Issue Mar 17, 2017 · 3 comments

Comments

Projects
None yet
2 participants
@microcreators

microcreators commented Mar 17, 2017

Environment:

  • Fail2Ban version (including any possible distribution suffixes):
    Fail2Ban v0.10.0a2
  • OS, including release name/version:
Linux 4.4.30-v7+ #919 SMP Tue Nov 1 16:57:28 GMT 2016 armv7l GNU/Linux
Raspbian GNU/Linux 8
  • Python:
    Python 2.7.9
  • Fail2Ban installed via OS/distribution mechanisms
  • You have not applied any additional foreign patches to the codebase
  • Some customizations were done to the configuration (provide details below is so)
    The original Debian distro version (0.8.x) has been overwritten by 0.10 cloned from github and installed. Old local config files stayed intact.

The issue:

I moved from the original Debian version 0.8.x to 0.10. I'm ignoring a few warnings about default values, but I see errors about HOST not defined. Which I'm assuming could be fine. Anyway... I think either the definitions are outdated or the interpreter. Maybe I mixed them...
See details below. Thanks for checking.

Steps to reproduce

Reboot or systemctl restart fail2ban. Check logs.

Expected behavior

Assuming jail definitions are fresh, the interpreter should not complain about the lack of HOST defined. Optionally, it should not complain about issues with disabled jails.

Observed behavior

fail2ban.log snippet:

017-03-17 21:50:42,813 fail2ban.filter [2320]: ERROR   No 'host' group in '^User <F-USER>.+</F-USER> not allowed because account is locked(?: \[preauth\])?\s*'
2017-03-17 21:50:42,818 fail2ban.filter [2320]: ERROR   No 'host' group in '^Disconnecting: Too many authentication failures(?: for <F-USER>.+?</F-USER>)?(?: \[preauth\])?\s*'
2017-03-17 21:50:42,833 fail2ban.filter [2320]: ERROR   No 'host' group in '<mdre-<mode>>'
2017-03-17 21:50:42,975 fail2ban.filter [2320]: ERROR   No 'host' group in '^client denied by server configuration: (uri )?\S*(, referer: \S+)?\s*$'
2017-03-17 21:50:42,979 fail2ban.filter [2320]: ERROR   No 'host' group in '^user .*? authentication failure for "\S*": Password Mismatch(, referer: \S+)?$'
2017-03-17 21:50:42,982 fail2ban.filter [2320]: ERROR   No 'host' group in '^user .*? not found(: )?\S*(, referer: \S+)?\s*$'
2017-03-17 21:50:42,986 fail2ban.filter [2320]: ERROR   No 'host' group in '^client used wrong authentication scheme: \S*(, referer: \S+)?\s*$'
2017-03-17 21:50:42,990 fail2ban.filter [2320]: ERROR   No 'host' group in '^Authorization of user \S+ to access \S* failed, reason: .*$'
2017-03-17 21:50:42,994 fail2ban.filter [2320]: ERROR   No 'host' group in '^([A-Z]\w+: )?user .*?: password mismatch: \S*(, referer: \S+)?\s*$'
2017-03-17 21:50:42,999 fail2ban.filter [2320]: ERROR   No 'host' group in '^([A-Z]\w+: )?user `.*?' in realm `.+' (not found|denied by provider): \S*(, referer: \S+)?\s*$'
2017-03-17 21:50:43,003 fail2ban.filter [2320]: ERROR   No 'host' group in '^user .*?: authorization failure for "\S*":(, referer: \S+)?\s*$'
2017-03-17 21:50:43,008 fail2ban.filter [2320]: ERROR   No 'host' group in '^([A-Z]\w+: )?invalid nonce .* received - length is not \S+(, referer: \S+)?\s*$'
2017-03-17 21:50:43,012 fail2ban.filter [2320]: ERROR   No 'host' group in '^([A-Z]\w+: )?realm mismatch - got `.*?' but expected `.+'(, referer: \S+)?\s*$'
2017-03-17 21:50:43,016 fail2ban.filter [2320]: ERROR   No 'host' group in '^([A-Z]\w+: )?unknown algorithm `.*?' received: \S*(, referer: \S+)?\s*$'
2017-03-17 21:50:43,020 fail2ban.filter [2320]: ERROR   No 'host' group in '^invalid qop `.*?' received: \S*(, referer: \S+)?\s*$'
2017-03-17 21:50:43,024 fail2ban.filter [2320]: ERROR   No 'host' group in '^([A-Z]\w+: )?invalid nonce .*? received - user attempted time travel(, referer: \S+)?\s*$'

Any additional information

The errors are mostly linked with sshd jail (and its different modes) and apache-auth, as both have multi-line definitions, not always with HOST defined.

Configuration, dump and another helpful excerpts

Relevant part of jail.local:

[ssh]
enabled  = true
port     = 111
filter   = sshd
logpath  = /var/log/auth.log
maxretry = 2
findtime = 300
bantime = 7200

[ssh-ddos]
enabled  = true
port     = 111
filter   = sshd-ddos
logpath  = /var/log/auth.log
maxretry = 5
findtime = 86400
bantime = 2592000

jail.conf:

[sshd]

# To use more aggressive sshd modes set filter parameter "mode" in jail.local:
# normal (default), ddos, extra or aggressive (combines all).
# See "tests/files/logs/sshd" or "filter.d/sshd.conf" for usage example and details.
mode    = normal
filter  = sshd[mode=%(mode)s]
port    = ssh
logpath = %(sshd_log)s
backend = %(sshd_backend)s

[apache-auth]

port     = http,https
logpath  = %(apache_error_log)s

sshd filter

# Fail2Ban filter for openssh
#
# If you want to protect OpenSSH from being bruteforced by password
# authentication then get public key authentication working before disabling
# PasswordAuthentication in sshd_config.
#
#
# "Connection from <HOST> port \d+" requires LogLevel VERBOSE in sshd_config
#

[INCLUDES]

# Read common prefixes. If any customizations available -- read them from
# common.local
before = common.conf

[DEFAULT]

_daemon = sshd

# optional prefix (logged from several ssh versions) like "error: ", "error: PAM: " or "fatal: "
__pref = (?:(?:error|fatal): (?:PAM: )?)?
# optional suffix (logged from several ssh versions) like " [preauth]"
__suff = (?: \[preauth\])?\s*
__on_port_opt = (?: port \d+)?(?: on \S+(?: port \d+)?)?

[Definition]

prefregex = ^<F-MLFID>%(__prefix_line)s</F-MLFID>%(__pref)s<F-CONTENT>.+</F-CONTENT>$

cmnfailre = ^[aA]uthentication (?:failure|error|failed) for <F-USER>.*</F-USER> from <HOST>( via \S+)?\s*%(__suff)s$
            ^User not known to the underlying authentication module for <F-USER>.*</F-USER> from <HOST>\s*%(__suff)s$
            ^Failed \S+ for (?P<cond_inv>invalid user )?<F-USER>(?P<cond_user>\S+)|(?(cond_inv)(?:(?! from ).)*?|[^:]+)</F-USER> from <HOST>%(__on_port_opt)s(?: ssh\d*)?(?(cond_user): |(?:(?:(?! from ).)*)$)
            ^<F-USER>ROOT</F-USER> LOGIN REFUSED.* FROM <HOST>\s*%(__suff)s$
            ^[iI](?:llegal|nvalid) user <F-USER>.*?</F-USER> from <HOST>%(__on_port_opt)s\s*$
            ^User <F-USER>.+</F-USER> from <HOST> not allowed because not listed in AllowUsers\s*%(__suff)s$
            ^User <F-USER>.+</F-USER> from <HOST> not allowed because listed in DenyUsers\s*%(__suff)s$
            ^User <F-USER>.+</F-USER> from <HOST> not allowed because not in any group\s*%(__suff)s$
            ^refused connect from \S+ \(<HOST>\)\s*%(__suff)s$
            ^Received disconnect from <HOST>%(__on_port_opt)s:\s*3: .*: Auth fail%(__suff)s$
            ^User <F-USER>.+</F-USER> from <HOST> not allowed because a group is listed in DenyGroups\s*%(__suff)s$
            ^User <F-USER>.+</F-USER> from <HOST> not allowed because none of user's groups are listed in AllowGroups\s*%(__suff)s$
            ^pam_unix\(sshd:auth\):\s+authentication failure;\s*logname=\S*\s*uid=\d*\s*euid=\d*\s*tty=\S*\s*ruser=<F-USER>\S*</F-USER>\s*rhost=<HOST>\s.*%(__suff)s$
            ^(error: )?maximum authentication attempts exceeded for <F-USER>.*</F-USER> from <HOST>%(__on_port_opt)s(?: ssh\d*)?%(__suff)s$
            ^User <F-USER>.+</F-USER> not allowed because account is locked%(__suff)s
            ^Disconnecting: Too many authentication failures(?: for <F-USER>.+?</F-USER>)?%(__suff)s
            ^<F-NOFAIL>Received disconnect</F-NOFAIL> from <HOST>: 11:
            ^<F-NOFAIL>Connection closed</F-NOFAIL> by <HOST>%(__suff)s$

mdre-normal =

mdre-ddos = ^Did not receive identification string from <HOST>%(__suff)s$
            ^Connection reset by <HOST>%(__on_port_opt)s%(__suff)s
            ^<F-NOFAIL>SSH: Server;Ltype:</F-NOFAIL> (?:Authname|Version|Kex);Remote: <HOST>-\d+;[A-Z]\w+:
            ^Read from socket failed: Connection reset by peer%(__suff)s

mdre-extra = ^Received disconnect from <HOST>%(__on_port_opt)s:\s*14: No supported authentication methods available%(__suff)s$
            ^Unable to negotiate with <HOST>%(__on_port_opt)s: no matching (?:cipher|key exchange method) found.
            ^Unable to negotiate a (?:cipher|key exchange method)%(__suff)s$

mdre-aggressive = %(mdre-ddos)s
                  %(mdre-extra)s

cfooterre = ^<F-NOFAIL>Connection from</F-NOFAIL> <HOST>

failregex = %(cmnfailre)s
            <mdre-<mode>>
            %(cfooterre)s

# Parameter "mode": normal (default), ddos, extra or aggressive (combines all)
# Usage example (for jail.local):
#   [sshd]
#   mode = extra
#   # or another jail (rewrite filter parameters of jail):
#   [sshd-aggressive]
#   filter = sshd[mode=aggressive]
#
mode = normal

#filter = sshd[mode=aggressive]

ignoreregex = 

maxlines = 1

journalmatch = _SYSTEMD_UNIT=sshd.service + _COMM=sshd

datepattern = {^LN-BEG}

# DEV Notes:
#
#   "Failed \S+ for .*? from <HOST>..." failregex uses non-greedy catch-all because
#   it is coming before use of <HOST> which is not hard-anchored at the end as well,
#   and later catch-all's could contain user-provided input, which need to be greedily
#   matched away first.
#
# Author: Cyril Jaquier, Yaroslav Halchenko, Petr Voralek, Daniel Black and Sergey Brester aka sebres
# Rewritten using prefregex (and introduced "mode" parameter) by Serg G. Brester.

sshd-ddos filter (no longer needed?)

# Fail2Ban ssh filter for at attempted exploit
#
# The regex here also relates to a exploit:
#
#  http://www.securityfocus.com/bid/17958/exploit
#  The example code here shows the pushing of the exploit straight after
#  reading the server version. This is where the client version string normally
#  pushed. As such the server will read this unparsible information as
#  "Did not receive identification string".

[INCLUDES]

# Read common prefixes. If any customizations available -- read them from
# common.local
before = common.conf

[Definition]

_daemon = sshd

failregex = ^%(__prefix_line)sDid not receive identification string from <HOST>\s*$

ignoreregex = 

# Author: Yaroslav Halchenko

apache-auth filter

# Fail2Ban apache-auth filter
#

[INCLUDES]

# Read common prefixes. If any customizations available -- read them from
# apache-common.local
before = apache-common.conf

[Definition]

prefregex = ^%(_apache_error_client)s (?:AH\d+: )?<F-CONTENT>.+</F-CONTENT>$

# auth_type = ((?:Digest|Basic): )?
auth_type = ([A-Z]\w+: )?

failregex = ^client denied by server configuration: (uri )?\S*(, referer: \S+)?\s*$
            ^user .*? authentication failure for "\S*": Password Mismatch(, referer: \S+)?$
            ^user .*? not found(: )?\S*(, referer: \S+)?\s*$
            ^client used wrong authentication scheme: \S*(, referer: \S+)?\s*$
            ^Authorization of user \S+ to access \S* failed, reason: .*$
            ^%(auth_type)suser .*?: password mismatch: \S*(, referer: \S+)?\s*$
            ^%(auth_type)suser `.*?' in realm `.+' (not found|denied by provider): \S*(, referer: \S+)?\s*$
            ^user .*?: authorization failure for "\S*":(, referer: \S+)?\s*$
            ^%(auth_type)sinvalid nonce .* received - length is not \S+(, referer: \S+)?\s*$
            ^%(auth_type)srealm mismatch - got `.*?' but expected `.+'(, referer: \S+)?\s*$
            ^%(auth_type)sunknown algorithm `.*?' received: \S*(, referer: \S+)?\s*$
            ^invalid qop `.*?' received: \S*(, referer: \S+)?\s*$
            ^%(auth_type)sinvalid nonce .*? received - user attempted time travel(, referer: \S+)?\s*$

ignoreregex = 

# DEV Notes:
#
# This filter matches the authorization failures of Apache. It takes the log messages
# from the modules in aaa that return HTTP_UNAUTHORIZED, HTTP_METHOD_NOT_ALLOWED or
# HTTP_FORBIDDEN and not AUTH_GENERAL_ERROR or HTTP_INTERNAL_SERVER_ERROR.
#
# An unauthorized response 401 is the first step for a browser to instigate authentication
# however apache doesn't log this as an error. Only subsequent errors are logged in the 
# error log.
#
# Source:
#
# By searching the code in http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/aaa/*
# for ap_log_rerror(APLOG_MARK, APLOG_ERR and examining resulting return code should get
# all of these expressions. Lots of submodules like mod_authz_* return back to mod_authz_core
# to return the actual failure.
#
# See also: http://wiki.apache.org/httpd/ListOfErrors
# Expressions that don't have tests and aren't common.
# more be added with  https://issues.apache.org/bugzilla/show_bug.cgi?id=55284 
#     ^%(_apache_error_client)s (AH01778: )?user .*: nonce expired \([\d.]+ seconds old - max lifetime [\d.]+\) - sending new nonce\s*$
#     ^%(_apache_error_client)s (AH01779: )?user .*: one-time-nonce mismatch - sending new nonce\s*$
#     ^%(_apache_error_client)s (AH02486: )?realm mismatch - got `.*' but no realm specified\s*$
#
# referer is always in error log messages if it exists added as per the log_error_core function in server/log.c
# 
# Author: Cyril Jaquier
# Major edits by Daniel Black and Sergey Brester (sebres)

@microcreators microcreators changed the title from no 'host' group error to no 'host' group error (default filters) Mar 17, 2017

@sebres

This comment has been minimized.

Show comment
Hide comment
@sebres

sebres Mar 17, 2017

Member

What do you see by start of 'fail2ban-client -V'
What do you see in fail2ban.log by "Starting Fail2ban v..."
If not Fail2Ban v0.10.0a2 then you've not clean installation (mix of the executable / fail2ban python modules / resp. wrong reference by init.d/fail2ban.service)

Anyway I think that something is not properly by your installation.

sshd-ddos filter (no longer needed?)

In the newest version you should use sshd filter with mode parameter set to ddos (or extra, or aggressive) .
The sshd-ddos filter is thus obsolete.

Member

sebres commented Mar 17, 2017

What do you see by start of 'fail2ban-client -V'
What do you see in fail2ban.log by "Starting Fail2ban v..."
If not Fail2Ban v0.10.0a2 then you've not clean installation (mix of the executable / fail2ban python modules / resp. wrong reference by init.d/fail2ban.service)

Anyway I think that something is not properly by your installation.

sshd-ddos filter (no longer needed?)

In the newest version you should use sshd filter with mode parameter set to ddos (or extra, or aggressive) .
The sshd-ddos filter is thus obsolete.

@microcreators

This comment has been minimized.

Show comment
Hide comment
@microcreators

microcreators Mar 17, 2017

It is telling me indeed it's Fail2Ban v0.10.0a2. But let me purge the previous version and reinstall 0.10 from here. I'll be back if the issue is still here... Thanks.

microcreators commented Mar 17, 2017

It is telling me indeed it's Fail2Ban v0.10.0a2. But let me purge the previous version and reinstall 0.10 from here. I'll be back if the issue is still here... Thanks.

@microcreators

This comment has been minimized.

Show comment
Hide comment
@microcreators

microcreators Mar 17, 2017

After detailed purging of the distro 0.8 release, all is good now. Thanks!

microcreators commented Mar 17, 2017

After detailed purging of the distro 0.8 release, all is good now. Thanks!

@sebres sebres closed this Mar 17, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment