--stdout and --stderr to support redirection to syslog

[dmolik]

I find myself supporting a lot of new Golang applications that don't have options to send logs to the syslog daemon. It would be great if I could handle that with a oneliner config change in an init script.

Perhaps it could be something like this:

--stdout syslog:daemon.info \
--stderr syslog:daemon.err

Where the tag would be populated by either $SVCNAME or --name.

I'm willing to write a PR for the change. I'm curious if there is community interest, and what the syntax/execution would be.

[dwfreed]

The only way to do this is with a separate process, which is attached to the daemon's stdout/stderr with pipes, and then just taking what comes along the pipes and submitting that to syslog. I think this is beyond the scope of start-stop-daemon.

[duanyao]

I'm interested in this feature, too. I'm developing a node.js application and have to send logs to local rsyslogd explicitly via TCP, so it lose some log system independency. If openrc could forward stdout/stderr to syslog, the application could be greatly simplified, and could also run under systemd+journald without modification.

I'm not a linux/unix expert, but using a separate log-forwarding process seems less reliable, isn't it? What if one of the processes connected to the pipes dies unexpectedly? Maybe the log-forwarding process itself need to be a openrc service, or even better, becomes part of openrc.

[dwfreed]

systemd already takes what's sent to syslog and puts it in the journal, if the journal is intended to take over for the usual syslog daemon. POSIX already defines a standard method for getting entries into the syslog, and this is exposed via the syslog() library call (which is part of the specification). The underlying interface is a Unix socket that lives at /dev/log; as such, it would be impossible to directly connect /dev/log to stdout/stderr, as syslog is expecting messages to be in a specific format (which the library call takes care of). As openrc has no persistent daemon of its own, a separate process, connected to the daemon to be redirected with pipes, would be needed. I believe this is beyond the scope of openrc.

Edit note: removed the remark about read()/write() not working on sockets, as a deeper inspection of their manpages indicate they do (though most people use send()/recv() anyway); note that there's still the problem of messages to /dev/log needing to be in a specific format.

[duanyao]

@dwfreed Thank for the detailed explanation!

As openrc has no persistent daemon of its own, a separate process, connected to the daemon to be redirected with pipes, would be needed.

I just tried supervise-daemon, isn't it such a persistent daemon that is potentially able to read the application's stdout and send to syslog?

For applications written in managed languages such as node.js apps, it is possible that it crashes in native codes and prints messages to stderr directly, and there is no easy way to send them to syslog by managed code. This is another reason why I think it's better for such apps to just log to stdout/stderr, and let the service manager to forward logs to syslog.

[dwfreed]

supervise-daemon is intended to be entirely analogous to start-stop-daemon with the exception that it monitors the child process, and restarts it if it exits. Adding syslog redirect capability to supervise-daemon would create differences between them that I personally don't want. Furthermore, it would add complication to the implementation, as there exists only one portable way to both listen for I/O and handle signals designed to change program state at the same time (namely, pselect), and it's a terrible API. This is much better implemented in a separate process, which listens on named pipes that you redirect the stdout and stderr of your daemon to, and i don't feel that this is something openrc needs to provide.

[duanyao]

Thanks for advice! Do you know any existing software which could do that (listens on named pipes that you redirect the stdout and stderr of your daemon to)?

[dmolik]

As you mentioned before, systemd.

[duanyao]

Well, I need to support Alpine Linux which doesn't have systemd out-of-box, so I decided to write a daemon myself to get the job done. It reads multiple named pipes and forward the contents to syslog(3), and other daemons just redirect their stdout|stderr to those named pipes. I would like to share it on github after some tests if anyone is interested.

[LW-archlinux]

Duanyao,
what you are describing sounds like it will be a new syslog-daemon that mimics systemd-journal behaviour .

Even if that is your intention, i'd check exisiting syslog daemon implementations like syslog-ng and rsyslog first.
Maybe they already support this ?

[duanyao]

Just checked the documents. Syslog-ng indeed supports collecting from named pipes, though I haven't used it.

Rsyslog (which I'm using currently) supports regular text file as input but seems not named pipe. Don't know how it behaves if the file is rotated.

[dmolik]

as a work around I'm considering dropping --stdout and --stderr in favor of pushing everything into exec option like this:

 (echo "command info"       |  logger --id=$$ -p daemon.info -t dan ) 2>&1 | logger --id=$$ daemon.err -t dan
 (echo "command error" 1>&2 |  logger --id=$$ -p daemon.info -t dan ) 2>&1 | logger --id=$$ daemon.err -t dan 

[vapier]

as @dwfreed documented, all of this seems out of scope for s-s-d. syslog works over a unix socket (or the syslog() function), and there's no way to directly connect them. any mechanism is either program specific (e.g. only working with rsyslog) or runs through a sep process (e.g. logger).

if we wanted to do something with openrc, it would involve creating a new daemon to connect fd's to, which would in turn take care of doing the read & write to syslog(). it would make resource utilization nicer (only one process to do select() and such), but would annoy some who think having any daemon being resident is "bad".

that said, i don't think having supervise-daemon manage this would be a bad idea considering it's already being kept resident specifically for the purpose of managing the child.

[dmolik]

I don't disagree with being out of scope with respect to s-s-d, but from a maintenance and rc-script writing perspective it seemed to be a convenient/logical place to put the configuration.

[kmoore134]

We would be highly interested in this functionality for TrueOS and possibly going into FreeBSD as well.

[williamh]

I am working on #127 currently, which is related. With it in place, you would be able to pipe to something like logger which could log for you. Does that cover what you need or are you looking at another type of functionality?

[kmoore134]

@williamh Yes, that would be exactly the kind of thing we'd want.

[LinkTed]

@dmolik I think you can close this issue because the feature is already merged in 0.37. So, the other user knows that the issue is resolved.

[dmolik]

I'll take a look, I'm not sure this was solved the way I thought would be most optimal.