Skip to content

install.sh: persist only what is ours — a firewall unit instead of iptables-persistent, scoped sysctl instead of --system#7

Merged
sweetcornna merged 1 commit into
mainfrom
install-scoped-persistence
Jul 17, 2026
Merged

install.sh: persist only what is ours — a firewall unit instead of iptables-persistent, scoped sysctl instead of --system#7
sweetcornna merged 1 commit into
mainfrom
install-scoped-persistence

Conversation

@sweetcornna

@sweetcornna sweetcornna commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

What & why

Fixes the two remaining defects from the end-to-end install test — both cases of a step reaching beyond what is Dormice's to change on an operator's host. Directions were agreed on the issues before this PR.

#4 — installing iptables-persistent silently apt-removes ufw. The metadata-firewall step now persists its two DROP rules with its own oneshot unit, dormice-metadata-firewall.service (After=/Requires=docker.service, Before=dormice.service, idempotent iptables -C || iptables -I), and stops installing iptables-persistent entirely. Beyond the ufw conflict, netfilter-persistent save always snapshotted the whole ruleset — operator rules and Docker's ephemeral NAT entries included — when exactly two rules are ours to persist. The unit is started at install time too: a unit that has never run is an unproven promise. Hosts persisted the old way keep their rules.v4 untouched; doctor's metadata-persisted check accepts either form.

#5sysctl --system replays operator config mid-install. The sysctl step (now "kernel parameters") applies scoped with sysctl -p /etc/sysctl.d/99-dormice.conf and proves boot-order effectiveness explicitly instead: systemd-sysctl --cat-config gives the real application order, and the winning value of each key must be the one we need — dying loudly with the offending file and value otherwise. Value, not file: a host may legitimately set ip_forward=1 in its own later-sorting config (the real test host does, via its VPN relay configs), and that is agreement, not a conflict. The scan also catches the /etc/sysctl.conf-via-99-sysctl.conf symlink that always outsorts 99-dormice.conf, which the old filename-sorting assumption missed. The file gains net.ipv4.ip_forward=1: dockerd only masks a boot config that says 0, so the hazard stays invisible until the next config replay cuts every sandbox's networking — exactly how the test install failed. A new ip-forward doctor check fails on effective 0 (sandboxes are cut right now) and warns, naming the file, when the boot winner is 0 behind an effective 1; it is the one check allowed to read boot config, because the effective value can never expose this hazard.

Closes #4. Closes #5.

Checklist

  • pnpm build && pnpm typecheck && pnpm lint && pnpm test passes locally (build first — the e2e suite runs the built daemon)
  • Behavior changes come with tests that fail without the change
  • User-facing changes to @dormice/shared / sdk / cli have a changeset (pnpm changeset)
  • README updated if this changes documented behavior (README needed no change — it names no persistence mechanism; docs/doctor on the website is updated)

Real-machine acceptance on the test host (from-scratch reinstall + loud-die negative test + reboot) in progress — results will be posted below before merge.

…tables-persistent, scoped sysctl instead of --system

The end-to-end install test (issues #4 and #5) caught the two steps
that reached beyond what is Dormice's to change on an operator's host.

The cloud-metadata step persisted its two DROP rules by installing
iptables-persistent — and apt resolves that package's conflict with
ufw by removing ufw, silently discarding the operator's firewall. The
netfilter-persistent snapshot was also always broader than our claim:
it saves the whole ruleset, operator rules and Docker's ephemeral NAT
entries included, when exactly two rules are ours to persist. Both
problems share one fix: a oneshot unit (dormice-metadata-firewall
.service), ordered after docker.service and before dormice.service,
re-adds the rules idempotently at boot — identical on every host, no
package dependency, no hand in anyone's firewall. It is started at
install time too, because a unit that has never run is an unproven
promise. Hosts persisted the old way keep their rules.v4 untouched;
doctor accepts either form and its fix line now points at install.sh.

The swappiness step applied its one-line file with `sysctl --system`,
which replays every sysctl source on the host mid-install. On the test
host that re-applied a dormant net.ipv4.ip_forward=0 from
/etc/sysctl.conf and cut Docker networking, surfacing two steps later
as an apt failure inside the base-image build with no visible
connection to sysctl. The step (now "kernel parameters") writes both
vm.swappiness=100 and net.ipv4.ip_forward=1 to 99-dormice.conf,
applies it scoped with sysctl -p, and proves boot-order effectiveness
explicitly instead: systemd-sysctl --cat-config yields the real
application order, and the last setter of each key must be our file —
which also catches the /etc/sysctl.conf-via-99-sysctl.conf symlink
that always outsorts 99-dormice.conf, and names the offending file and
value when it dies.

ip_forward gains persistence because dockerd only masks the problem:
it flips the flag on at every daemon start, so a boot config saying 0
stays invisible until the next config replay cuts every sandbox's
networking. That is also why the new doctor check is the one allowed
to read boot config — the effective value can never expose this
hazard. Effective 0 fails (sandboxes are cut right now); effective 1
with a boot winner of 0 warns and names the file.

Closes #4. Closes #5.
@sweetcornna
sweetcornna force-pushed the install-scoped-persistence branch from 1841301 to 64a2a8d Compare July 17, 2026 15:59
@sweetcornna

Copy link
Copy Markdown
Collaborator Author

Real-machine acceptance on the Debian 12 test host (the same machine both issues were found on) — all green, at 64a2a8d:

  1. From-scratch install from this branch, one run. Old persistence purged first (iptables-persistent + /etc/iptables removed, both DROP rules deleted from the live chain, base image removed, all Dormice state wiped). Install completed with dor doctor: 21 passed, 0 failed, 0 warnings, 3 skipped — including the sandbox base-image build that issue install.sh: sysctl --system replays operator config mid-install and can cut Docker networking #5's replay used to kill. The new check reads honestly: net.ipv4.ip_forward = 1 — 1 (effective value, persisted by /etc/sysctl.d/99-mlvpn-relay.conf).
  2. The winner scan is value-based, not file-based — refined during acceptance. This host legitimately sets ip_forward=1 in its own later-sorting configs (99-mlvpn-relay.conf); requiring the last setter to be our file would have died on a healthy host. The check now requires the winning value to be the needed one — a later file re-stating our value is agreement, not a conflict. (Doctor already judged by value.)
  3. Negative test — the incident replayed. With net.ipv4.ip_forward = 0 restored to /etc/sysctl.conf, the installer exits 1 at the kernel-parameters step: net.ipv4.ip_forward must be 1 at boot, but the sysctl boot order ends with /etc/sysctl.d/99-sysctl.conf setting it to 0 — change or remove that line, then re-run. And because the apply is scoped, the operator's 0 was never applied — effective value stayed 1 throughout, where the old code would have cut Docker networking mid-install.
  4. Idempotent re-run: every step [skip], the winner scan still verifies, doctor 21/0 again.
  5. Reboot. ActiveEnterTimestampMonotonic proves the declared order: docker 8.324s → dormice-metadata-firewall 8.362s → dormice 8.365s. The unit re-added both DROP rules into the fresh DOCKER-USER chain (no rules.v4 exists anymore to do it), doctor 21/0 after boot, and the host's own services (nginx, tailscaled, fail2ban, xray, open-websearch) all came back.
  6. Smoke test: acquire → exec (uname -r = 4.19.0-gvisor) → destroy, all clean.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant