Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] - refresh-policy-routes systemd timer emits lots of noisy journald log entries #62

Closed
wilsonzlin opened this issue Jul 12, 2022 · 3 comments

Comments

@wilsonzlin
Copy link

The refresh-policy-routes systemd timer unit, which appears to trigger the refresh-policy-routes systemd service unit (which itself runs /usr/bin/setup-policy-routes), runs around once every 2 minutes, which adds up to a lot of noisy repeated log messages in journald, making it harder to scan the logs and find our important system and application log messages.

systemd itself emits many messages due to the starting and ending of various services (including itself), and the setup-policy-routes script also emits debug logs that are mostly unnecessary. Here's what shows up in our logs every 2 minutes:

Starting refresh-policy-routes@ens5.service - Set up policy routes for ens5...
Got IMDSv2 token from http://169.254.169.254/latest
Starting configuration for ens5
/lib/systemd/systemd-networkd-wait-online ens5
[get_meta] Querying IMDS for mac
Configuring ens5 as primary
Using existing cfgfile /run/systemd/network/70-ens5.network
[get_meta] Querying IMDS for network/interfaces/macs/XX:XX:XX:XX:XX:XX/local-ipv4s
refresh-policy-routes@ens5.service: Deactivated successfully.
Finished refresh-policy-routes@ens5.service - Set up policy routes for ens5.
refresh-policy-routes@ens5.service: Consumed 31ms CPU time, received 1.5K IP traffic, sent 1.3K IP traffic.

I can think of a few possible solutions, not sure if applicable:

  • Run the script/program as a long running systemd service, instead of invoking repeatedly on a timer, to stop the repeated messages from systemd itself about the lifecycle of the service.
  • Only emit warning/error messages from the script, not debug/info messages, by default.

In the meantime, is it safe to disable this service? I'm not sure what it does and I couldn't find any documentation/references online or a description in the script itself, and I haven't had time to dive deep into the code yet.

@nmeyerhans
Copy link
Contributor

If you will not be assigning any additional secondary IPv4 addresses or delegated prefixes to your ENIs, then you can stop and/or disable the timer.

We can consider reducing some of the logs to debug severity in order to simplify filtering.

I'm going to transfer this issue to the amazon-ec2-net-utils repository and we can track work on it there.

@nmeyerhans nmeyerhans transferred this issue from amazonlinux/amazon-linux-2023 Jul 12, 2022
@nmeyerhans
Copy link
Contributor

I've reduced the verbosity of some of the messages over the past couple of releases. Resolving this.

@yan12125
Copy link

yan12125 commented Nov 15, 2023

@nmeyerhans I still get many logs in journald. Should I open a new issue or could you reopen this issue? I need additional IPv4 addresses, so I cannot disable refresh-policy-routes.

My current workaround is reducing the systemd log level and hijacking the logger command:

# /etc/systemd/system/refresh-policy-routes@.service.d/silent.conf
[Service]
# Avoid frequent messages from ec2net.
LogLevelMax=notice
BindReadOnlyPaths=/usr/bin/true:/usr/bin/logger:norbind

Update: it seems BindReadOnlyPaths= is needed due to a systemd limitation systemd/systemd#2913

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants