Log encoding=auto ends up being ANSI_X3.4-1968

[szepeviktor]

# fail2ban-client get recidive logencoding
Current log encoding is set to:
ANSI_X3.4-1968

Fail2ban v0.9.5 is started by SysVinit.

Should I add LC_ALL="en_US.UTF-8" to /etc/defaults/fail2ban to set auto logencoding to UTF-8?
The Debian system locale is en_US.UTF-8.

Please advise.

[szepeviktor]

Logged in as root:

$ python3
Python 3.4.2 (default, Oct  8 2014, 10:45:20)
[GCC 4.9.1] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale
>>> print(locale.getpreferredencoding())
UTF-8

[sebres]

No idea why...
Still acute issue?
Only recidive, or another jails affected also?

[szepeviktor]

I have not backported 0.9.6.
In 0.9.5-1 it is still ANSI_X3.4-1968

$ fail2ban-client get recidive logencoding
Current log encoding is set to:
ANSI_X3.4-1968

[sebres]

recidive only?

[szepeviktor]

Excuse me.
No.

$ fail2ban-client get apache-instant logencoding
Current log encoding is set to:
ANSI_X3.4-1968

[sebres]

Could you provide results of:

python -c "from fail2ban.server.filter import Filter; f = Filter(None); print(f.getLogEncoding())"

fail2ban-client -d | grep -E "(recidive|apache).*encoding"

No idea about default encoding by services starting via SysVinit, but I think it not matters normally, and should use the system encoding (at least I've never seen something others).

[szepeviktor]

Here it is

cd /usr/lib/python3/dist-packages/
python -c "from fail2ban.server.filter import Filter; f = Filter(None); print(f.getLogEncoding())"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "fail2ban/server/filter.py", line 38, in <module>
    from .action import CommandAction
  File "fail2ban/server/action.py", line 109
    class ActionBase(object, metaclass=ABCMeta):
                                      ^
SyntaxError: invalid syntax

fail2ban-client -d | grep -E "(recidive|apache).*encoding"
['set', 'recidive', 'logencoding', 'auto']
['set', 'apache-default', 'logencoding', 'auto']
['set', 'apache-combined', 'logencoding', 'auto']
['set', 'apache-instant', 'logencoding', 'auto']
['set', 'apache-cf1-combined', 'logencoding', 'auto']
['set', 'apache-cf1-instant', 'logencoding', 'auto']

[szepeviktor]

Excuse me, Fail2ban runs on Python 3

python3 -c "from fail2ban.server.filter import Filter; f = Filter(None); print(f.getLogEncoding())"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
AttributeError: 'Filter' object has no attribute 'getLogEncoding'

[szepeviktor]

This is the env of /usr/bin/fail2ban-server

BOOT_IMAGE=/boot/vmlinuz-3.16.0-4-amd64
COLUMNS=80
CONSOLE=/dev/console
HOME=/root
INIT_VERSION=sysvinit-2.88
LINES=25
PATH=/usr/sbin:/usr/bin:/sbin:/bin
PREVLEVEL=N
PWD=/
RUNLEVEL=2
SHELL=/bin/sh
SHLVL=3
TERM=linux
_=/sbin/start-stop-daemon
drop_caps=
init=/sbin/init
previous=N
rootmnt=/root
runlevel=2

[szepeviktor]

But 😄

python3 -c "import locale; print(locale.getpreferredencoding())"
UTF-8

[szepeviktor]

But I think on the terminal it comes from LC_ALL=en_US.UTF-8

And fail2ban-server has no such env variable.

[sebres]

IMHO the debian way is to set locale as LANG and not LC_ALL...

And corresponding this man sysvinit-utils/service.8, service should pass LANG (as well as LC_ALL) to the calling init script.

What did you have in /etc/default/locale (resp. in /etc/environment, but no idea about this in sysvinit)?

Check the locales are installed and select system locale via dpkg-reconfigure...

?sudo? apt-get install locales
?sudo? dpkg-reconfigure locales

Which env has your init process? (cat /proc/1/environ)

[szepeviktor]

cat /proc/1/environ|tr '\0' '\n'
SHLVL=1
HOME=/
init=/sbin/init
TERM=linux
drop_caps=
BOOT_IMAGE=/boot/vmlinuz-3.16.0-4-amd64
PATH=/sbin:/usr/sbin:/bin:/usr/bin
PWD=/
rootmnt=/root


cat /etc/default/locale
#  File generated by update-locale
LANG=en_US.UTF-8

/etc/environment is empty as in all Debian

[sebres]

Fixed on the code-level in #1868, but ...

Note: Notwithstanding, this fixes many encoding problems, would be a good idea correctly set all expected environment variables for fail2ban daemon in init- resp. systemd-scripts or server configuration.