Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

--stdout and --stderr to support redirection to syslog #118

Open
dmolik opened this issue Feb 27, 2017 · 18 comments
Open

--stdout and --stderr to support redirection to syslog #118

dmolik opened this issue Feb 27, 2017 · 18 comments

Comments

@dmolik
Copy link

dmolik commented Feb 27, 2017

I find myself supporting a lot of new Golang applications that don't have options to send logs to the syslog daemon. It would be great if I could handle that with a oneliner config change in an init script.

Perhaps it could be something like this:

--stdout syslog:daemon.info \
--stderr syslog:daemon.err

Where the tag would be populated by either $SVCNAME or --name.

I'm willing to write a PR for the change. I'm curious if there is community interest, and what the syntax/execution would be.

@dwfreed
Copy link
Contributor

dwfreed commented Mar 4, 2017

The only way to do this is with a separate process, which is attached to the daemon's stdout/stderr with pipes, and then just taking what comes along the pipes and submitting that to syslog. I think this is beyond the scope of start-stop-daemon.

@duanyao
Copy link

duanyao commented May 4, 2017

I'm interested in this feature, too. I'm developing a node.js application and have to send logs to local rsyslogd explicitly via TCP, so it lose some log system independency. If openrc could forward stdout/stderr to syslog, the application could be greatly simplified, and could also run under systemd+journald without modification.

I'm not a linux/unix expert, but using a separate log-forwarding process seems less reliable, isn't it? What if one of the processes connected to the pipes dies unexpectedly? Maybe the log-forwarding process itself need to be a openrc service, or even better, becomes part of openrc.

@dwfreed
Copy link
Contributor

dwfreed commented May 4, 2017

systemd already takes what's sent to syslog and puts it in the journal, if the journal is intended to take over for the usual syslog daemon. POSIX already defines a standard method for getting entries into the syslog, and this is exposed via the syslog() library call (which is part of the specification). The underlying interface is a Unix socket that lives at /dev/log; as such, it would be impossible to directly connect /dev/log to stdout/stderr, as syslog is expecting messages to be in a specific format (which the library call takes care of). As openrc has no persistent daemon of its own, a separate process, connected to the daemon to be redirected with pipes, would be needed. I believe this is beyond the scope of openrc.

Edit note: removed the remark about read()/write() not working on sockets, as a deeper inspection of their manpages indicate they do (though most people use send()/recv() anyway); note that there's still the problem of messages to /dev/log needing to be in a specific format.

@duanyao
Copy link

duanyao commented May 4, 2017

@dwfreed Thank for the detailed explanation!

As openrc has no persistent daemon of its own, a separate process, connected to the daemon to be redirected with pipes, would be needed.

I just tried supervise-daemon, isn't it such a persistent daemon that is potentially able to read the application's stdout and send to syslog?

For applications written in managed languages such as node.js apps, it is possible that it crashes in native codes and prints messages to stderr directly, and there is no easy way to send them to syslog by managed code. This is another reason why I think it's better for such apps to just log to stdout/stderr, and let the service manager to forward logs to syslog.

@dwfreed
Copy link
Contributor

dwfreed commented May 4, 2017

supervise-daemon is intended to be entirely analogous to start-stop-daemon with the exception that it monitors the child process, and restarts it if it exits. Adding syslog redirect capability to supervise-daemon would create differences between them that I personally don't want. Furthermore, it would add complication to the implementation, as there exists only one portable way to both listen for I/O and handle signals designed to change program state at the same time (namely, pselect), and it's a terrible API. This is much better implemented in a separate process, which listens on named pipes that you redirect the stdout and stderr of your daemon to, and i don't feel that this is something openrc needs to provide.

@duanyao
Copy link

duanyao commented May 4, 2017

Thanks for advice! Do you know any existing software which could do that (listens on named pipes that you redirect the stdout and stderr of your daemon to)?

@dmolik
Copy link
Author

dmolik commented May 8, 2017

As you mentioned before, systemd.

@duanyao
Copy link

duanyao commented May 9, 2017

Well, I need to support Alpine Linux which doesn't have systemd out-of-box, so I decided to write a daemon myself to get the job done. It reads multiple named pipes and forward the contents to syslog(3), and other daemons just redirect their stdout|stderr to those named pipes. I would like to share it on github after some tests if anyone is interested.

@LW-archlinux
Copy link

Duanyao,
what you are describing sounds like it will be a new syslog-daemon that mimics systemd-journal behaviour .

Even if that is your intention, i'd check exisiting syslog daemon implementations like syslog-ng and rsyslog first.
Maybe they already support this ?

@duanyao
Copy link

duanyao commented May 9, 2017

Just checked the documents. Syslog-ng indeed supports collecting from named pipes, though I haven't used it.

Rsyslog (which I'm using currently) supports regular text file as input but seems not named pipe. Don't know how it behaves if the file is rotated.

@dmolik
Copy link
Author

dmolik commented Feb 9, 2018

as a work around I'm considering dropping --stdout and --stderr in favor of pushing everything into exec option like this:

 (echo "command info"       |  logger --id=$$ -p daemon.info -t dan ) 2>&1 | logger --id=$$ daemon.err -t dan
 (echo "command error" 1>&2 |  logger --id=$$ -p daemon.info -t dan ) 2>&1 | logger --id=$$ daemon.err -t dan 

@vapier
Copy link
Member

vapier commented Feb 10, 2018

as @dwfreed documented, all of this seems out of scope for s-s-d. syslog works over a unix socket (or the syslog() function), and there's no way to directly connect them. any mechanism is either program specific (e.g. only working with rsyslog) or runs through a sep process (e.g. logger).

if we wanted to do something with openrc, it would involve creating a new daemon to connect fd's to, which would in turn take care of doing the read & write to syslog(). it would make resource utilization nicer (only one process to do select() and such), but would annoy some who think having any daemon being resident is "bad".

that said, i don't think having supervise-daemon manage this would be a bad idea considering it's already being kept resident specifically for the purpose of managing the child.

@dmolik
Copy link
Author

dmolik commented Feb 11, 2018

I don't disagree with being out of scope with respect to s-s-d, but from a maintenance and rc-script writing perspective it seemed to be a convenient/logical place to put the configuration.

@kmoore134
Copy link

We would be highly interested in this functionality for TrueOS and possibly going into FreeBSD as well.

@williamh
Copy link
Contributor

williamh commented Jun 8, 2018

I am working on #127 currently, which is related. With it in place, you would be able to pipe to something like logger which could log for you. Does that cover what you need or are you looking at another type of functionality?

@kmoore134
Copy link

@williamh Yes, that would be exactly the kind of thing we'd want.

@LinkTed
Copy link
Contributor

LinkTed commented Jun 12, 2021

@dmolik I think you can close this issue because the feature is already merged in 0.37. So, the other user knows that the issue is resolved.

@dmolik
Copy link
Author

dmolik commented Jun 16, 2021

I'll take a look, I'm not sure this was solved the way I thought would be most optimal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants