Skip to content

Fail2Ban Exim WARNING fail2ban.log (since version 0.9.0) #718

Closed
dave670 opened this Issue May 7, 2014 · 2 comments

2 participants

@dave670
dave670 commented May 7, 2014

Jail.conf:

# "logencoding" specifies the encoding of the log files handled by the jail
#   This is used to decode the lines from the log file.
#   Typical examples:  "ascii", "utf-8"
#
#   auto:   will use the system locale setting
logencoding = auto

logencoding = ASCII and logencoding= UTF-8 have this problem

Exim filter (exim.conf):

# Fail2Ban filter for exim
#
# This includes the rejection messages of exim. For spam and filter
# related bans use the exim-spam.conf
#

[INCLUDES]

# Read common prefixes. If any customizations available -- read them from
# exim-common.local
before = exim-common.conf

[Definition]

failregex = ^%(pid)s %(host_info)ssender verify fail for <\S+>: (?:Unknown user|Unrouteable address|all relevant MX records point to non-existent hosts)\s*$
             ^%(pid)s \w+ authenticator failed for (\S+ )?\(\S+\) \[<HOST>\]: 535 Incorrect authentication data( \(set_id=.*\)|: \d+ Time\(s\))?\s*$
             ^%(pid)s %(host_info)sF=(<>|[^@]+@\S+) rejected RCPT [^@]+@\S+: (relay not permitted|Sender verify failed|Unknown user)\s*$
             ^%(pid)s SMTP protocol synchronization error \([^)]*\): rejected (connection from|"\S+") %(host_info)s(next )?input=".*"\s*$
             ^%(pid)s SMTP call from \S+ \[<HOST>\](:\d+)? (I=\[\S+\]:\d+ )?dropped: too many nonmail commands \(last was "\S+"\)\s*$

ignoreregex =

# DEV Notes:
# The %(host_info) defination contains a <HOST> match
#
# SMTP protocol synchronization error \([^)]*\)  <- This needs to be non-greedy
# to void capture beyond ")" to avoid a DoS Injection vulnerabilty as input= is
# user injectable data.
#
# Author: Cyril Jaquier
#         Daniel Black (rewrote with strong regexs)

/var/log/exim/mainlog:

mail subject: T="the price is € 100.00"
mail subject exim: T="the price is â~B¬100.00"

Problem Fail2Ban (/var/log/fail2ban.log):

2014-05-06 17:15:27,998 fail2ban.server.filter[95535]: WARNING Error decoding line from '/var/log/exim/mainlog' with 'US-ASCII': '2014-05-06 16:26:56 1WhgKd-000OMD-Gn <=  H= [] P=esmtps X=id= T="\xd0\xa1\xd0\xb5\xd0\xbc\xd0\xb8\xd0\xbd\xd0\xb0\xd1\x80 \xd0\xbf\xd0\xbe \xd0\xbf\xd1\x80\xd0\xb0\xd0\xba\xd1\x82\xd0\xb8\xd0\xba\xd0\xb5 \xd0\xba\xd0\xb0\xd0\xb4\xd1\x80\xd0\xbe\xd0\xb2\xd0\xbe\xd0\xb3\xd0\xbe \\n =?utf-8?Q?=D0=B4=D0=B5=D0=BB=D0=BE=D0=BF=D" from <> for 

fail2ban.conf:

# Fail2Ban main configuration file
#
# Comments: use '#' for comment lines and ';' (following a space) for inline comments
#
# Changes:  in most of the cases you should not modify this
#           file, but provide customizations in fail2ban.local file, e.g.:
#
# [Definition]
# loglevel = DEBUG
#

[Definition]

# Option: loglevel
# Notes.: Set the log level output.
#         CRITICAL
#         ERROR
#         WARNING
#         NOTICE
#         INFO
#         DEBUG
# Values: [ LEVEL ]  Default: ERROR
#
loglevel = NOTICE

# Option: logtarget
# Notes.: Set the log target. This could be a file, SYSLOG, STDERR or STDOUT.
#         Only one log target can be specified.
#         If you change logtarget from the default value and you are
#         using logrotate -- also adjust or disable rotation in the
#         corresponding configuration file
#         (e.g. /etc/logrotate.d/fail2ban on Debian systems)
# Values: [ STDOUT | STDERR | SYSLOG | FILE ]  Default: STDERR
#
logtarget = /var/log/fail2ban.log

system:

  • FreeBSD 9.2
  • Exim 4.82
  • Fail2Ban 0.9.0 (version 0.8.12 does not have this problem)

How can I fix this problem?

@kwirk
Fail2Ban member
kwirk commented May 7, 2014

I've tried to tidy up the issue text, which I've hopefully broken into the correct blocks :smile:


Fail2Ban 0.9.x supports Python 3, and hence the handling of encoding is required due to Python 3 differentiation between bytes and strings. This is why you don't see the issue 0.8.12.

The line is still being processed fully in Python 2, and in Python 3 is merely drops the characters it can't decode. Maybe the warning message should be clearer that this is the case…

The example error from fail2ban.log appears to be valid UTF-8:

In [1]: b"\xd0\xa1\xd0\xb5\xd0\xbc\xd0\xb8\xd0\xbd\xd0\xb0\xd1\x80 \xd0\xbf\xd0\xbe \xd0\xbf\xd1\x80\xd0\xb0\xd0\xba\xd1\x82\xd0\xb8\xd0\xba\xd0\xb5 \xd0\xba\xd0\xb0\xd0\xb4\xd1\x80\xd0\xbe\xd0\xb2\xd0\xbe\xd0\xb3\xd0\xbe".decode('utf-8')
Out[1]: 'Семинар по практике кадрового'
@kwirk
Fail2Ban member
kwirk commented May 15, 2014

@dave670: Just to add, you can set the logencoding per jail if that helps.

If there is an issue with these warnings spamming the Fail2Ban log file, maybe there needs to be an option to suppress these warnings?

I'll close for now and hopefully the clearer message in #723 will help.

@kwirk kwirk closed this May 15, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.