Skip to content

3. Rollback e2e + hardening

Omisen edited this page Aug 14, 2026 · 4 revisions

The step from “it works in the tests” to “ready for a real machine”. It adds no dangerous features — it adds guarantees and settles debts. It lives in tests/rollback_e2e.rs, src/logging.rs, src/lockfile.rs, and in the two TODOs it closes.

It closes the circle opened at the beginning: the promise “either the installation succeeds, or the system goes back to exactly what it was” becomes a verified property, not an intention.


End-to-end rollback (the heart, G7)

Until here each undo was tested in isolation. This tests the rollback's emergent property over the whole sequence: inject a failure at step N and check that the 0..N-1 chain undoes itself, bringing the system back to its pre-installation state.

SystemModel: a stateful model of the system

Unlike the mock that records operations, SystemModel models a coherent state (users, packages, services enabled/active, roles and databases, files, symlinks and their contents, firewall rules). Every mutation updates the state; every undo restores it; all the steps of a sequence share the same model. So “back to pristine” is verifiable literally:

final_state == initial_state

The scenarios

Scenario What it demonstrates
Full chain + a failure at the end the whole sequence rolls back → identical state; the .bashrc byte for byte
Failure halfway (after user/deps/postgres/role/DB) everything that had been created disappears
The customer's pre-existing resources the init hard stop halts the chain; the rollback touches nothing pre-existing

The pre-existing resources scenario is the proof of the three protections as a property of the whole chain, not of a single step: starting from a system with PostgreSQL installed and running and an odoo database that already exists (the customer's data), after the rollback:

  • the customer's database is NOT dropped (anti-drop);
  • PostgreSQL stays installed (D3) and the already-running service stays running (D4);
  • a pre-existing /opt/odoo stays.

That is the moment the installer becomes safe to point at a real machine.


Logging to a file (G1)

tracing with two layers: TTY (with colours, coordinated with the progress bar) and a file without ANSI (/var/log/invok.log, non-blocking writer). It captures the whole run (INFO and above), errors and rollback included — the artifact to ask for in a post-mortem. It degrades to TTY-only if the path is not writable (a dry-run without root), without failing. The password never gets there, guaranteed by Secret.

Since R7 the log lives in /var/log, outside /opt/odoo: it is opened before the engine, so keeping it inside would create the very directory the first step is supposed to create — and would keep it occupied at the last undo.


The concurrency lock (G5)

An exclusive non-blocking flock (through nix) on /run/invok.lock, with an RAII guard: the release happens on success, on error and on panic (Drop). A second simultaneous run gets a clear error — “another installation is in progress”without mutating anything. It is acquired after the preflight checks and before any mutation, and skipped in dry-run.

acquire deliberately does not create its parent directory: taking a lock is coordination, and it must not bring artifacts into existence (R7).


Closing the two TODOs

wkhtmltopdf checksums — the TOFU decision (trust-on-first-use)

The official wkhtmltopdf/packaging release publishes neither checksums nor signatures for its packages (only the git tag is GPG-signed). There is therefore no upstream checksum to use.

The honest decision: manual TOFU pinning. The fail-closed mechanism of 1.5 is unchanged; the default_checksums() table is populated with pins generated once from a trusted source (official HTTPS). From then on the installer verifies every download against that pin — protecting against compromised mirrors, corrupted downloads and later alterations, even without an upstream signature. They are TOFU pins, not official checksums: the documentation says so, along with the procedure to generate and update them. A suffix with no pin refuses the installation — honest behaviour, never bypassed.

PostgreSQL cluster caution

The TODO from 1.6 SetupPostgres: even with --aggressive-rollback, before purging PostgreSQL it is checked (best-effort) that the cluster hosts no other databases besides ours. If it finds any → purge declined with a warning (stop and disable still applied); if the list cannot be obtained → fail-safe, nothing is purged. The decision is taken before the stop, because listing databases requires a running postgres.


Design notes

  • The lock and the logging live in main and dedicated modules; the Step trait and the engine stay untouched.
  • The end-to-end tests run entirely on SystemModel — no real system, no root.
  • What mock tests cannot prove is covered by the integration CI, which really installs and uninstalls: see Development and contributing.

Clone this wiki locally