systemd-journal() source doesn't work in official docker images

[akunszt]

I'm trying to use the balabit/syslog-ng:3.14.1 to read from the local journald. This is working in a docker image I created from debian:stretch by installing syslog-ng-core and syslog-ng-mod-journal. I have other issues with the Debian based image (the syslog-ng crashes every time in certain situations, but that's another issue) and I liked to check if the official one crashes or not as the Debian one is only 3.8.

I use this command line:
docker run -it --rm -h "syslog-$RANDOM" -v ~/syslog-ng.conf-3.14:/etc/syslog-ng/syslog-ng.conf -v /tmp/dq:/var/lib/syslog-ng -v /var/log/journal:/var/log/journal -v /run/log:/run/log -v /run/systemd:/run/systemd -v /etc/machine-id:/etc/machine-id balabit/syslog-ng:3.14.1 --no-caps -edv

The syslog-ng.conf-3.14 is a minimal one:

@version: 3.14

options {
        chain-hostnames( off );
        flush-lines( 0 );
        log-msg-size( 65536 );
        time-zone( "UTC" );
        ts-format( iso );
        use-dns( no );
        use-fqdn( no );
};

source s_docker_journald {
        internal();
        systemd-journal( prefix( ".SDATA.journal." ) );
};

rewrite r_fakehost {
        set( "KISMACSKA", value( ".SDATA.journal._HOSTNAME" ) );
};

destination d_relay {
        syslog(
                "172.17.0.1"
                transport( "tcp" )
                port( 1234 )
                log-fifo-size( 1000 )
                disk-buffer(
                        reliable( no )
                        disk-buf-size( 1073741824 )
                        mem-buf-length( 10000 )
                )
        );
};

log {
        source( s_docker_journald );
        rewrite( r_fakehost );
        destination( d_relay );
};

The server is a plain netcat. The first run doesn't shows any usable debug information:

[2018-03-08T13:17:52.219061] Seeking the journal to the start position;
[2018-03-08T13:17:52.219215] Processing the time zone file (32bit part); filename='/usr/share/zoneinfo/UTC'
[2018-03-08T13:17:52.219271] Processing the time zone file (32bit part); filename='/usr/share/zoneinfo/UTC'
[2018-03-08T13:17:52.219915] Disk-buffer state saved; filename='/var/lib/syslog-ng/syslog-ng-00000.qf', qout_length='0', qbacklog_length='0', qoverflow_length='0', qdisk_length='0'

At the second run it complains that the position stored in the persist file for the journal driver is empty. The OSE edition doesn't contains the persist-tool, so I can't check the syslog-ng.persist contents. If you wish I can send the file for you.

[2018-03-08T13:18:10.680943] Failed to seek journal to the saved cursor position; cursor='', error='Invalid argument (22)'
[2018-03-08T13:18:10.680976] Seeking the journal to the start position;
[2018-03-08T13:18:10.681121] Processing the time zone file (32bit part); filename='/usr/share/zoneinfo/UTC'
[2018-03-08T13:18:10.681172] Processing the time zone file (32bit part); filename='/usr/share/zoneinfo/UTC'
[2018-03-08T13:18:10.681482] Disk-buffer state loaded; filename='/var/lib/syslog-ng/syslog-ng-00000.qf', qout_length='0', qbacklog_length='0', qoverflow_length='0', qdisk_length='0'

The very same docker command with the very same syslog-ng.conf works with the plain Debian based image.

[akunszt]

Please check syslog-ng/syslog-ng#1919 (comment) for the details. The systemd-journal() source doesn't work on newer systems (systemd versions), but it's work on Ubuntu 16.04.

It might worth to add another docker image based on Debian 9.0 or Ubuntu 17.10.

Also it might worth to mention that the user has to "cross-mount" the journald storages: -v /var/log/journal:/run/log/journal. It's not intuitive so many users should miss it at the first time.

[MrAnno]

We've upgraded the Docker image to use recent versions of Debian.
Please open a new issue in the syslog-ng/syslog-ng repo if you still experience this problem.