Skip to content

feat(checks): a check name is a category, not an instance - #480

Merged
passcod merged 1 commit into
mainfrom
feat/check-names-are-categories
Aug 4, 2026
Merged

feat(checks): a check name is a category, not an instance#480
passcod merged 1 commit into
mainfrom
feat/check-names-are-categories

Conversation

@passcod

@passcod passcod commented Aug 4, 2026

Copy link
Copy Markdown
Member

🤖 Stacked on #479 — review that first; this branch is based on it because both touch the same files.

Why

Backup checks spelled the backup type into the name: backup-staleness:tamanu-postgres, backup-staleness:caddy-config, backup-staleness:tamanu-config, backup-staleness:postgres-config. A deployment backing up four things produced four checks to configure, four catalog rows, and four entries in every listing — for one condition. That defeats the point of a catalog you configure once and then use to get your eyes on things quickly.

The rule

A check name is a category. Anything that varies between instances of the same condition — which backup configuration, which restore intent — goes in the check's detail, where policy rules already read it as check.<field>.

New normative sections: CHK#names (a name is a category, instances go in detail, and how an instanced check aggregates), its application in BKJ, and the AGENTS.md bullet that landed with #479.

How it aggregates

The five backup sweeps — staleness, never, reconcile-missing, report-gap, size-mismatch — now file one check per server with the types as instances:

  • each instance is graded through policy on its own, against its own detail, so a rule or silence written for one backup type still affects only that type. That was the whole reason the type was in the name.
  • the check settles on its most urgent instance;
  • its detail carries every degraded instance with that instance's own result, and its message names them.

So an operator configures staleness once for the fleet, and where a particular type warrants different treatment, writes check.type == '…' rather than acquiring another check.

New in issues.rs: file_check_instances, with file_check reduced to its one-instance case so nothing else changes behaviour. The catalog entry and scoped chain are read once per check rather than once per instance, so a server with a dozen backup types still costs two queries.

Two details worth a look

Silences. A skipped instance leaves the aggregate rather than counting as healthy, and a check whose instances are all skipped is itself skipped. But the observed result still spans every instance including the silenced ones — silencing changes what canopy acts on, never what it saw, which CHK requires. Getting this wrong initially broke scoped_check_policies::server_silence_grades_filings_to_skipped, which is a good test.

Undecidable is not resolved. A stale repo inventory makes the missing verdict undecidable. Where it is stale for every type there is nothing to conclude, so an already-open finding is left alone instead of being cleared on the strength of a lagging inspector. That is easy to lose in a refactor, so there is a test for it.

Migration

Collapses the stored state three ways:

  • issues — per-type rows onto one per target. The survivor is the most urgent, then the most recently seen, so it keeps the row an operator is most likely already looking at along with its incident membership and history; losers leave their incidents first, and incidents left with no failing contributor retire.
  • silences (scoped policies) — a silence on backup-staleness:tamanu-postgres becomes a rule guard on check.type, so operators keep exactly the suppression they configured instead of it widening to every type or being dropped. Verified to produce the correct if-ladder: {"if": [{"==": [{"var": "check.type"}, "tamanu-postgres"]}, "skipped"]}, with pre-existing rules preserved behind the guard.
  • catalog — per-type rows onto one, preferring an operator-reviewed policy over a canopy-seeded one.

All three verified against synthetic data in a rolled-back transaction.

Deliberately not in scope

restore-verification, redaction, and migration-test still carry (type, intent) in their names. A follow-up is in progress.

Backup checks spelled the backup type into the name:
backup-staleness:tamanu-postgres, backup-staleness:caddy-config, and so
on. A deployment backing up four things produced four checks to
configure, four catalog rows, and four entries in every listing — for
one condition. That defeats the point of a catalog you configure once
and then use to get your eyes on things quickly.

The five backup sweeps now file one check per server with the types as
instances. Each instance is graded through policy on its own, against
its own detail, so an operator writing a rule or a silence for one
backup type still affects only that type — the reason the type was in
the name in the first place. The check then settles on its most urgent
instance, carries every degraded instance in its detail with that
instance's own result, and names them in its message.

New in issues.rs: file_check_instances, with file_check reduced to its
one-instance case so nothing else changes behaviour. The catalog entry
and scoped chain are read once per check rather than once per instance,
so a server with a dozen backup types still costs two queries.

Skipped instances (a silence matched) leave the aggregate rather than
counting as healthy, and a check whose instances are all skipped is
itself skipped. The observed result still spans every instance including
the silenced ones: silencing changes what canopy acts on, never what it
saw, which the CHK spec requires.

One behaviour subtlety kept deliberately: a stale repo inventory makes
the missing verdict undecidable, not resolved. Where it is stale for
every type there is nothing to conclude, so an already-open finding is
left alone instead of being cleared on the strength of a lagging
inspector. Covered by a test, since it is easy to lose.

The rule is now normative: a new Names section in the CHK spec (a name
is a category; instances go in detail; how an instanced check
aggregates), its application in BKJ, and the AGENTS.md bullet added
alongside the severity rule.

The migration collapses the stored state: per-type issue rows onto one
per target (survivor is the most urgent, then most recently seen, and
losers leave their incidents so emptied ones retire), per-type silences
into check.type rule guards so operators keep exactly the suppression
they configured rather than having it widen to every type or vanish, and
per-type catalog rows onto one, preferring an operator-reviewed policy
over a canopy-seeded one.

restore-verification, redaction, and migration-test still carry
(type, intent) in their names. They are filed from an ingest path rather
than a sweep, so the filer never holds the full instance set; routing
them through a shared recompute-per-server is a restructure of the
restore subsystem and belongs in its own change.
Base automatically changed from fix/backup-checks-warn-by-default to main August 4, 2026 03:07
@passcod
passcod added this pull request to the merge queue Aug 4, 2026
Merged via the queue into main with commit 5333d31 Aug 4, 2026
7 checks passed
@passcod
passcod deleted the feat/check-names-are-categories branch August 4, 2026 04:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant