Fails to start if destination host unreachable

[algernon]

Reported by one of the Debian admins, syslog-ng 3.5.6 fails to start if a TLS destination is unreachable at the time syslog-ng is started during boot. This is very unfortunate, because this makes the whole system unbootable without manual intervention.

The desired behaviour would be suspending the destination until time-reopen() or somesuch. As far as I remember, something similar was attempted for tcp() destinations, but it appears it still does not work.

[ihrwein]

I can confirm, that syslog-ng v3.5.6 is indeed affected, but I couldn't reproduce the bug with v3.6.

We can find and correct this bug but it would require a new Debian syslog-ng release (3.5.7). Since 3.6 isn't affected, we can use it directly to create unofficial packages that can be given to you. What do you think about it?

[algernon]

I'd prefer a 3.5.7, or a fix on the 3.5 branch, with which I can patch the official Debian package. Since this would go on official Debian machines, it would need to be in Jessie, which will ship with 3.5.

[ihrwein]

I tried to reproduce this issue with a clean install dir, but it solved the problem. I found a very similar issue (maybe the same), at [1]. Could you upload to somewhere an strace log? I suspect there is a different version from libsyslog-ng.so somewhere in the linker path.

[1] https://bugzilla.balabit.com/show_bug.cgi?id=226

[algernon]

There definitely isn't any conflicting libsyslog-ng.so, the Debian dependencies make that impossible. The problem is different too: it's not related to iv_tls_user_register at all. It fails to start because syslog-ng fails to initialize the destination, and syslog-ng itself decides that it should quit. It's not a crash, but a fail-to-start scenario.

I don't have the resources to reproduce it at the moment, but I was told that with TLS, this happens with 100% reliability (as long as the destination is, indeed, not resolvable).

[ihrwein]

So, if syslog-ng cannot lookup the destination's domain name (because the DNS record doesn't exist) it fails to start? Or it cannot connect to the destination (because of firewalls, network outage, etc)?

[algernon]

If it can't look it up, or if it can't connect, then it fails to start, correct. The desired behaviour would be to start, with the destination suspended, and retried after time-reopen(). Basically, the same behaviour that can be observed when a destination disconnects, should be applied to startup too.

[hbakken]

I can reproduce this in 3.6.2 as well, and for both syslog(... transport("udp")) and syslog(... transport("tls"))

This problem prevents us from forwarding logs using syslog-ng directly, so we have to resort to piping via a program() driver or a local pipe or something.

[czanik]

There is a related bug also in the openSUSE bugzilla at https://bugzilla.opensuse.org/show_bug.cgi?id=921252
Please fix it also for syslog-ng 3.5 (current openSUSE also has this version).

[czanik]

The opensuse ticket has some further info and workarounds added. Might be just another expression of the same problem or something completely different, but the end result is the same: syslog-ng does not try hard enough to reach network destinations and fails.

[faxm0dem]

For the record we stumbled on something similar with a standard network() destination

[justcallmegreg]

PR #490 hopefully solves this issue. (Merged)

[ihrwein]

@algernon @gregory094 : What should be the done definition of this issue? I think the problem is solved by the PR.

[justcallmegreg]

The problem stated above is solved. I think we can close this issue.