Remote maintenance

Remote maintenance

Run any action on another host over SSH — same zipapp transport as the scanner (nothing installed on the target; it just needs python3 and the same fettle version). The grammar is:

fettle remote [--ssh-arg ARG]... HOST [any fettle action/flags...]

Everything after HOST is forwarded verbatim to fettle on the remote, so the full CLI works remotely. Changes are wrapped in sudo (the remote fettle runs as root); a --dry-run needs neither sudo nor a password.

fettle remote server1                  # safe default: clean + update + firmware-check
fettle remote server1 -c -u            # clean, then upgrade packages
fettle remote server1 update --dry-run # preview an update; changes nothing
fettle remote server1 -a --yes         # the full default set, unattended
fettle remote server1 -S               # security scan on the host (sys-audit --all)
fettle remote server1 upgrade-check    # AI pre-upgrade check (analysed on YOUR box)
fettle remote server1 orphans kernel   # destructive actions run only when named
fettle remote --ssh-arg=-oConnectTimeout=5 server1 -u

• Safe by default. fettle remote <host> with no action named runs only clean update firmware-check — even with --yes. Destructive/interactive actions (orphan and kernel removal) run only when you name them; -a forwards through and runs the remote's full default set.
• upgrade-check (-U) analyses locally. fettle remote <host> upgrade-check is special: fettle collects a (redacted) snapshot on the remote — read-only, no sudo, no API key — and runs the AI analysis on your machine with your local key. Your key never leaves your machine, only your machine needs internet to Anthropic, and the report is saved locally as ~/.fettle/reports/<host>/. (On Debian the remote's pending list is read from cached apt data, so it may be stale if the host hasn't apt updated recently; Arch uses a fresh rootless sync.)
• Asks before upgrading. By default the run is interactive over an ssh -t TTY: the remote package manager shows its plan and prompts before upgrading (and sudo prompts for a password if needed). This is the same locally — fettle -u asks; it does not auto-upgrade.
• --yes = fully unattended. No prompts at all: pacman --noconfirm / apt-get … --force-confold full-upgrade -y (keeps old conffiles), no TTY. It assumes passwordless sudo on the target, and on Arch it skips yay's PKGBUILD review — only use it on hosts whose sources you trust.

After an unattended (--yes) run, review kept config files with fettle remote <host> config-drift (apt keeps the old file and drops a .dpkg-dist; pacman leaves a .pacnew).

Host groups

Define a group of hosts in the config and run on all of them, in order, with one command — e.g. update the whole lab:

# ~/.config/fettle/config.toml
[remote.groups.bifrost-lab]
hosts    = ["bifrost", "ec1", "ec2", "ec3"]   # ~/.ssh/config aliases, hostnames, or IPs
# actions  = ["-a"]        # optional: default action(s) when none given on the CLI
# ssh_args = ["-o", "ConnectTimeout=5"]   # optional: merged with any CLI --ssh-arg
# yes      = true          # optional: always run unattended

[remote.groups]            # shorthand — a bare list is {hosts = [...]}
arch-boxes = ["mjolnir", "wopr"]

fettle remote bifrost-lab -a          # run `fettle -a` on each host, in order
fettle remote bifrost-lab -a --yes    # unattended (no confirm; needs passwordless sudo)

fettle remote <group> runs the same per-host flow on each host sequentially. It confirms the host list before starting (skipped under --yes / --dry-run), continues past a host that fails, and prints a pass/fail summary at the end (the command exits non-zero if any host failed). A group name takes precedence over a same-named single host; an unknown name is treated as a single host. One group (or host) per command.

After each host's run, fettle fetches back that host's reports and its own run-log (the session transcript, including the package-update output) into ~/.fettle/{reports,logs}/<host>/. So in the HTML report, a group run shows up as a per-host entry under each target host — not as a single "group" asset. The "group runs" area itself is just a one-line pass/fail summary of each orchestration.

For a truly walk-away group run, use --yes (or yes = true) and set up passwordless sudo (NOPASSWD) on the group's hosts — otherwise each host stops for its sudo password over the interactive ssh -t.

On Debian/Ubuntu, fettle keeps the apt upgrade from opening a full-screen ncurses dialog (needrestart's service-restart menu, debconf config screens) — those corrupt the terminal over ssh -t. needrestart only lists what needs restarting (restart later with sudo needrestart); debconf prompts, if any, are plain text.

A standalone binary (for hosts with no python3 at all) is a planned option; the zipapp is the current transport. It's uploaded to the remote user's home under a random name (not a predictable world-writable /tmp path) and removed after the run.