Support encoding on glib compiled with libiconv

[bkil-syslogng]

If the user compiles glib with libiconv on a platform without a native iconv, character encoding can not be specified. (The issue is not easy to reproduce on many systems, but LD_PRELOAD should do the trick.)

Given the following config:

@version: 3.8

options
{ threaded(yes); }
;
source s_program_37b57ebaaa6a460085d2d35d6f7d56ec
{ program("cat" encoding("UTF-8")); }
;
destination d_file_a3219214c5e541d5a18b0e9ca3eedee2
{ file("/tmp/81525975-13e5-4a41-8e9e-1bbd7cdc3d25.txt"); }
;
log
{ source(s_program_37b57ebaaa6a460085d2d35d6f7d56ec); destination(d_file_a3219214c5e541d5a18b0e9ca3eedee2); flags(flow-control); };

Startup fails with the following message:

[2016-05-10T11:51:22.050595] Module loaded and initialized successfully; module='afprog'
Error parsing afprog, unknown encoding UTF-8 in /home/tamasnagy/install/opt/syslog-ng/etc/syslog-ng.conf at line 7, column 88:

{ program("/bin/cat /home/tamasnagy/install/opt/syslog-ng/etc/syslog-ng.conf" encoding("UTF-8")); }
                                                                                       ^^^^^^^

syslog-ng documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
mailing list: https://lists.balabit.hu/mailman/listinfo/syslog-ng

It worked correctly on syslog-ng 3.7.3.

The root cause is that lib/logproto/logproto-server.c checks supported encodings by first trying to convert from "utf8" to the given encoding. However, this always fails because "utf8" is not considered a valid encoding on some of the libraries. Internal canonicalization is done by upper casing each character.

On a separate note, this analysis concludes that the user should be advised that only those codes can be used in a syslog-ng config which are supported by the used iconv implementation, i.e., giving { program("cat" encoding("UTF8")); } should also fail on such a system. syslog-ng does not implement an extra aliasing layer and glib's aliasing doesn't seem to be exhaustive in this regard either.

Here are the aliases for libiconv:

• http://git.savannah.gnu.org/cgit/libiconv.git/tree/lib/encodings.def?id=48f31c744eb14da699c6ba3fd03f5f232c476bbb#n60

Here are some of the aliases for glib:

• https://github.com/GNOME/glib/blob/master/glib/libcharset/config.charset

[bkil-syslogng]

As a further enhancement, should we also add an utf-8 alias here?

• https://github.com/balabit/syslog-ng/blob/f36548bd805772d3709319b39c4ac7801e69d267/lib/logmatcher.c#L876

[bazsi]

I dont think so. The value in logmatcher is a pcre flag and not an encoding
value.
On May 10, 2016 12:13 PM, "Tamas Nagy" notifications@github.com wrote:

As a further enhancement, should we also add an utf-8 alias here?

https://github.com/balabit/syslog-ng/blob/f36548bd805772d3709319b39c4ac7801e69d267/lib/logmatcher.c#L876

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#1048 (comment)

[bkil-syslogng]

Well, as we see, some people prefer writing utf-8, while others prefer utf8. Providing an alias here could reduce confusion.

[bazsi]

There's an alias: unicode. I dont mind the dashed version either.
On May 12, 2016 11:40 AM, "Tamas Nagy" notifications@github.com wrote:

Well, as we see, some people prefer writing utf-8, while others prefer
utf8. Providing an alias here could reduce confusion.

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
#1048 (comment)