feat(restore): one restore check per server, replicas as instances

[passcod]

🤖 Completes the check-name work started in #480. Those three checks were the remaining parameterised names.

Why

restore-verification, redaction and migration-test were filed under names carrying (type, intent) — restore-verification:tamanu-postgres:verify. Both dimensions are open-ended strings (BackupType::Custom(String), RestoreIntent(pub String)), so the catalog fanned out without bound: three conditions became three × types × intents entries to configure. Worse than the backup case #480 fixed, which had one open dimension.

They are now one check each per server, with each replica an instance graded on its own. Type, intent and the replica's name live in the filing's detail, where policy rules reach them.

The derivation rule

This is the part worth reviewing. A replica key (server, type, intent) yields an instance when any of:

1. An enabled declaration covers it on that server — whatever the consumer currently advertises. Capability and semantic checks were removed from instance existence and kept only where they're genuinely needed, computing overdue. An unadvertised intent is a gap to surface, not grounds to drop a standing finding.
2. A report exists for it and an enabled declaration still asks for that replica somewhere in the server's group. So a report about a server the declaration doesn't name still surfaces against that server.
3. A recorded migration verdict exists for it, ungated — so a verdict with replica_id: None and no declaration anywhere still surfaces.

An instance's result is the worse of its latest report and its overdue bound. Those used to be two writers racing on one check name.

Two bugs fell out of getting this right: a check that is open but derives no instances is now recovered rather than left open forever when the last replica goes away, and servers with open checks that derive nothing are visited at all.

Where the agent's brief was wrong, and it said so

I briefed this as "a key with a report but no declaration still yields an instance". That cannot coexist with the four tests whose guarantees I also said must hold. Under the literal rule, a deleted or disabled declaration leaves a failed report as a permanently derivable instance that nothing can clear — the ingest only accepts reports an enabled declaration authorizes, so once nothing declares the replica, no report can ever arrive to recover it. Rule 2 above is the closest thing to the intent that doesn't manufacture unrecoverable findings.

Two fixtures changed as a consequence, neither by loosening an assertion:

• three record_report_* tests now declare the replica they report on — a report with no declaration at all is a state production cannot reach, so those fixtures described an impossible world;
• re_enabling_does_not_recover_anything records its report while the declaration is enabled, then disables, then re-enables. Its middle assertion contradicted disabling_recovers_the_stale_alert head-on; a stateless re-derivation cannot distinguish a report that landed while the replica was decommissioned from one that landed before. The guarantee in the name still holds.

The two-parameter silence guard does not fit the DSL

Condition is one operator over one variable and IfLadder has no and/or, so check.type == X AND check.intent == Y is inexpressible. Widening a silence to every replica of a type, or dropping it, were both unacceptable — an operator's stored suppression has to survive exactly. So every instance carries check.replica_key = "<type>:<intent>" and the migration guards on that. Composite identity in the detail, not in the name, so the rule from #480 holds. Documented in the migration, the sweep, CHK, and the shipped check documentation.

Migration

collapse_restore_check_names mirrors #480's across the three prefixes: issues collapse onto one survivor per target (losers leave their incidents; emptied incidents retire), silences become check.replica_key guards, catalog rows collapse preferring an operator-reviewed one. Verified against synthetic data — two servers, colliding pairs, an incident held only by losers, a pre-existing bare migration-test catalog row, and an unrelated colon-bearing check — in a rolled-back transaction.

Operator-visible

• One check of each kind per server; stored silences rewritten to per-replica guards.
• A finding appears or clears on the next sweep (minute cadence) rather than at ingest. These are non-paging warnings, so the delay is immaterial; the gain is one filer instead of two that drift.
• A withdrawn capability no longer silently clears a finding. A deleted, disabled or rescoped declaration still does.
• A failed migration test raises the check with no declaration in sight — which it did via the ingest path before, and briefly stopped doing mid-change.
• Servers with declarations but no findings no longer get passing rows or catalog entries seeded until something degrades.

Follow-up noted, not done

latest_by_key and latest_verdict_by_key are fleet-wide DISTINCT ON scans each pass. The existing index is (server_id, type, observed_at DESC), so Postgres sorts rather than walking it. Fine at current volume; an index on (server_id, type, intent, observed_at DESC) is the obvious fix if backup_restore_checks grows.

Also: two declarations covering one key (group-wide plus server-scoped) produce one instance labelled by the first by (name, id), with their overdue verdicts OR'd.

Commits

The first commit is deliberately the broken intermediate — it derives instances from declarations only — with the fix layered on top rather than squashed in, so the two are revertible apart. Its message says which part is wrong.