Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

journald sends messages to syslog even when configured not to forward them #15807

Closed
nomis opened this issue May 13, 2020 · 4 comments · Fixed by #15812
Closed

journald sends messages to syslog even when configured not to forward them #15807

nomis opened this issue May 13, 2020 · 4 comments · Fixed by #15812
Labels
journal pid1

Comments

@nomis
Copy link
Contributor

nomis commented May 13, 2020 •
edited
Loading

systemd version the issue has been seen with
237 (but the code is the same in 245: https://github.com/systemd/systemd/blob/v245/src/journal/journald-stream.c#L272)

Used distribution
Ubuntu 18.04.4

Expected behaviour you didn't see
journald is configured not to forward to syslog, so it never sends messages to /run/systemd/journal/syslog.

Unexpected behaviour you saw
If a service is configured to output to syslog, journald sends messages to /run/systemd/journal/syslog regardless of the configuration.

Steps to reproduce the problem

  1. Configure journald with ForwardToSyslog=no
  2. Configure a service that sends stdout to syslog:
[Service]
StandardOutput=syslog
  1. Configure syslog to read from the journal and ignore (do not read) /run/systemd/journal/syslog
  2. Ensure that syslog.socket is running
  3. Start the service so that it outputs (lots of) messages

Eventually the socket buffer fills up and journald will complain about missing messages sent to syslog.

Most service configuration is in distribution packages so it's not practical to try and override their output settings.
@nomis
Copy link
Contributor Author

nomis commented May 13, 2020

If syslog.socket is not running, all the calls to sendmsg return ENOENT and it ignores the missing message. I don't think journald should be sending messages that will never go anywhere when it is configured not to.

I've tracked down why syslog.socket is running to an event where syslog-ng was misconfigured and stopped responding:

2020-05-02T18:44:19.024+01:00 root@papylos systemd[1]: Stopping System Logger Daemon...
2020-05-02T18:45:49.073+01:00 root@papylos systemd[1]: syslog-ng.service: State 'stop-sigterm' timed out. Killing.
2020-05-02T18:45:49.075+01:00 root@papylos systemd[1]: syslog-ng.service: Killing process 743 (syslog-ng) with signal SIGKILL.
2020-05-02T18:45:49.093+01:00 root@papylos systemd[1]: syslog-ng.service: Main process exited, code=killed, status=9/KILL
2020-05-02T18:45:49.093+01:00 root@papylos systemd[1]: syslog-ng.service: Failed with result 'timeout'.
2020-05-02T18:45:49.093+01:00 root@papylos systemd[1]: Stopped System Logger Daemon.
2020-05-02T18:45:49.094+01:00 root@papylos systemd[1]: Listening on Syslog Socket.
2020-05-02T18:45:49.099+01:00 root@papylos systemd[1]: Starting System Logger Daemon...
2020-05-02T18:45:49.186+01:00 root@papylos systemd[1]: Started System Logger Daemon.

Stopping syslog.socket when it shouldn't be running stops spurious messages like systemd-journald[32640]: Forwarding to syslog missed 30 messages. from being generated.

@poettering
Copy link
Member

StandardOutput=syslog is long obsolete, we don't document it anymore even. it enables forwarding to syslog even when journald wouldn't do that normally.

Simply don#t use StandardOutput=syslog. The implied service default is StandardOutput=journal anyway, which does not suffer by this problem.

poettering added a commit to poettering/systemd that referenced this issue May 14, 2020
@poettering
core: automatically update StandardOuput=syslog to =journal
dbcc0c5 
Let's go one step further and upgrade implicitly. Usually =syslog
assignments are historic artifacts only. Let's upgrade the lines
automatically, and politely suggest people update their unit
files/configuration.

Fixes: systemd#15807
@poettering poettering mentioned this issue May 14, 2020
Merged
@poettering
Copy link
Member

#15812 gets rid of the concept, and implicitly patches StandardOutput=syslog to mean StandardOutput=journal to avoid this issue altogether

@poettering poettering reopened this May 14, 2020
@poettering
Copy link
Member

oops. pressed the wrong button. let's github auto-close the issue when the PR gets merged.

poettering added a commit to poettering/systemd that referenced this issue May 14, 2020
@poettering
core: automatically update StandardOuput=syslog to =journal
441d7c3 
Let's go one step further and upgrade implicitly. Usually =syslog
assignments are historic artifacts only. Let's upgrade the lines
automatically, and politely suggest people update their unit
files/configuration.

Fixes: systemd#15807
poettering added a commit to poettering/systemd that referenced this issue May 14, 2020
@poettering
core: automatically update StandardOuput=syslog to =journal (and simi…
f3dc6af 
…lar for StandardError=)

Let's go one step further and upgrade implicitly. Usually =syslog
assignments are historic artifacts only. Let's upgrade the lines
automatically, and politely suggest people update their unit
files/configuration (and drop the lines altogether, without
replacement).

Fixes: systemd#15807
hramrach added a commit to hramrach/ppc64-diag that referenced this issue Mar 19, 2021
@hramrach
Drop obsolete logging options from systemd service files.
dda27d6 
See systemd/systemd#15807 LTC#192095 bsc#1183700

Signed-off-by: Michal Suchanek <msuchanek@suse.de>
@hramrach hramrach mentioned this issue Mar 19, 2021
Merged
smortex added a commit to smortex/go-choria that referenced this issue Sep 2, 2021
@smortex
Fix deprecation warnings
5482971 
Recent systemd include a warning message when StandardOutput /
StandardError is set to "syslog" in a unit file [15812].

Quoting [poettering] from another PR which caused him to create this
one:
> StandardOutput=syslog is long obsolete, we don't document it anymore
> even. it enables forwarding to syslog even when journald wouldn't do
> that normally.
>
> Simply don#t use StandardOutput=syslog. The implied service default is
> StandardOutput=journal anyway, which does not suffer by this problem.

So simply remove these lines and honor the DefaultStandardOutput /
DefaultStandardError if the admin set them.

[15812]: systemd/systemd#15812
[poettering]: systemd/systemd#15807 (comment)
smortex added a commit to smortex/go-choria that referenced this issue Sep 2, 2021
@smortex
Fix deprecation warnings
3c5fd81 
Recent systemd include a warning message when StandardOutput /
StandardError is set to "syslog" in a unit file [15812].

Quoting [poettering] from another PR which caused him to create this
one:
> StandardOutput=syslog is long obsolete, we don't document it anymore
> even. it enables forwarding to syslog even when journald wouldn't do
> that normally.
>
> Simply don#t use StandardOutput=syslog. The implied service default is
> StandardOutput=journal anyway, which does not suffer by this problem.

So simply remove these lines and honor the DefaultStandardOutput /
DefaultStandardError if the admin set them.

[15812]: systemd/systemd#15812
[poettering]: systemd/systemd#15807 (comment)
smortex added a commit to smortex/go-choria that referenced this issue Sep 2, 2021
@smortex
(choria-io#1388) Fix deprecation warnings
ce64703 
Recent systemd include a warning message when StandardOutput /
StandardError is set to "syslog" in a unit file [15812].

Quoting [poettering] from another PR which caused him to create this
one:
> StandardOutput=syslog is long obsolete, we don't document it anymore
> even. it enables forwarding to syslog even when journald wouldn't do
> that normally.
>
> Simply don#t use StandardOutput=syslog. The implied service default is
> StandardOutput=journal anyway, which does not suffer by this problem.

So simply remove these lines and honor the DefaultStandardOutput /
DefaultStandardError if the admin set them.

[15812]: systemd/systemd#15812
[poettering]: systemd/systemd#15807 (comment)
smortex added a commit to smortex/go-choria that referenced this issue Sep 2, 2021
@smortex
(choria-io#1388) Fix deprecation warnings
0f0f91b 
Recent systemd include a warning message when StandardOutput /
StandardError is set to "syslog" in a unit file [15812].

Quoting [poettering] from another PR which caused him to create this
one:
> StandardOutput=syslog is long obsolete, we don't document it anymore
> even. it enables forwarding to syslog even when journald wouldn't do
> that normally.
>
> Simply don#t use StandardOutput=syslog. The implied service default is
> StandardOutput=journal anyway, which does not suffer by this problem.

So simply remove these lines and honor the DefaultStandardOutput /
DefaultStandardError if the admin set them.

[15812]: systemd/systemd#15812
[poettering]: systemd/systemd#15807 (comment)
joostruis added a commit to mocaccinoOS/desktop that referenced this issue Nov 19, 2021
@joostruis
apps/lightdm: Drop obsolete StandardOutput=syslog
c45ebad 
See systemd/systemd#15807
@lgrn lgrn mentioned this issue Apr 30, 2024
Merged
mdecimus pushed a commit to stalwartlabs/mail-server that referenced this issue May 5, 2024
@lgrn
Remove StandardOutput, StandardError in service (#390)
0d78f66 
systemd 249 (used in Ubuntu 22.04 LTS) throws the following error:

/etc/systemd/system/stalwart-mail.service:16: Standard output type
syslog is obsolete, automatically updating to journal. Please update
your unit file, and consider removing the setting altogether.

/etc/systemd/system/stalwart-mail.service:17: Standard output type
syslog is obsolete, automatically updating to journal. Please update
your unit file, and consider removing the setting altogether.

This is caused by StandardOutput=syslog and StandardError=syslog found
in install.sh, which are as the error suggests obsolete options that
should not be used.[1] They have "correct" settings in
stalwart-mail.service, but these settings are also the implicit
default, which means they are not necessary.[1]

This was patched in systemd #15812 in May, 2020.[2]

I have tested this by simply removing the values in
/etc/systemd/system/stalwart-mail.service on my server, running
'daemon-reload' and restarting the service, then confirming that logs
are still being written to journalctl as well as /opt/stalwart-mail/logs

[1]: <systemd/systemd#15807 (comment)>
[2]: <systemd/systemd#15812>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
journal pid1
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Get rid of StandardOutput=syslog poettering/systemd
2 participants
@nomis @poettering