Skip to content

Add possibility to restart systemd services through annotations#220

Draft
majst01 wants to merge 6 commits into
masterfrom
annotation-controllers
Draft

Add possibility to restart systemd services through annotations#220
majst01 wants to merge 6 commits into
masterfrom
annotation-controllers

Conversation

@majst01
Copy link
Copy Markdown
Contributor

@majst01 majst01 commented May 27, 2026

Description

This PR allows operators and users to restart systemd services running on the firewall. For users there is a small whitelist of services, which are allowed to be restarted.

This contributes to the headscale upgrade path to version >= v0.27.1 allowing operators to easily re-connect the tailscale clients on the firewalls.

Sample:

k annotate firewall -n namespace <firewall-name> firewall.metal-stack.io/restart-systemd-services=droptailer

TODO:

  • Annotation somehow not removed after restart
  • Add Documentation

Used AI-Tools ✨

  • None used for generation

@majst01 majst01 changed the title Add possibility to restart services on Firewall through annotations Add possibility to restart systemd services through annotations May 27, 2026
@majst01
Copy link
Copy Markdown
Contributor Author

majst01 commented May 27, 2026

Nope:

May 27 12:34:58 shoot--pbs4kr--skifoan0-firewall-9ebd8 ip[112935]: {"time":"2026-05-27T12:34:58.43324249+02:00","level":"ERROR","msg":"unable to create firewall annotation controller","error":"controller with name firewall already exists. Controller names must be unique to avoid multiple controllers reporting the same metric. This validation can be disabled via the SkipNameValidation option"}
May 27 12:34:58 shoot--pbs4kr--skifoan0-firewall-9ebd8 ip[112935]: panic: controller with name firewall already exists. Controller names must be unique to avoid multiple controllers reporting the same metric. This validation can be disabled via the SkipNameValidation option
May 27 12:34:58 shoot--pbs4kr--skifoan0-firewall-9ebd8 ip[112935]: goroutine 1 [running]:
May 27 12:34:58 shoot--pbs4kr--skifoan0-firewall-9ebd8 ip[112935]: main.main()
May 27 12:34:58 shoot--pbs4kr--skifoan0-firewall-9ebd8 ip[112935]:         ./main.go:303 +0x2ba5

@majst01
Copy link
Copy Markdown
Contributor Author

majst01 commented May 27, 2026

Now after annotating the firewall object in the seed:

$ k annotate -n shoot--pbs4kr--skifoan0 firewall shoot--pbs4kr--skifoan0-firewall-9ebd8 firewall.metal-stack.io/restart-systemd-services=droptailer
metal@shoot--pbs4kr--skifoan0-firewall-9ebd8:~$ sudo journalctl -lfu firewall-controller | grep -i drop
May 27 12:48:13 shoot--pbs4kr--skifoan0-firewall-9ebd8 ip[114529]: {"time":"2026-05-27T12:48:13.603232326+02:00","level":"INFO","msg":"restart service","logger":"controllers/FirewallAnnotation","service-name":"droptailer.service"}
^C
metal@shoot--pbs4kr--skifoan0-firewall-9ebd8:~$ systemctl status droptailer
● droptailer.service - Droptailer
     Loaded: loaded (/etc/systemd/system/droptailer.service; enabled; preset: enabled)
     Active: active (running) since Wed 2026-05-27 12:48:13 CEST; 13s ago
   Main PID: 114789 (droptailer-clie)
      Tasks: 7 (limit: 38019)
     Memory: 4.0M (peak: 5.3M)
        CPU: 49ms
     CGroup: /system.slice/droptailer.service
             └─vrf
               └─vrf50
                 └─114789 /usr/local/bin/droptailer-client

but forcing a second service to restart gives:

$ k annotate -n shoot--pbs4kr--skifoan0 firewall shoot--pbs4kr--skifoan0-firewall-9ebd8 firewall.metal-stack.io/restart-systemd-services=tailscaled
error: --overwrite is false but found the following declared annotation(s): 'firewall.metal-stack.io/restart-systemd-services' already has a value (droptailer)

also tested from a shoot against the firewall-monitor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants