Skip to content

[infra] Evaluate org rulesets as the scaling mechanism for review enforcement #90

Description

@twistedmelonman

Follow-up from the W0 protection pass (2026-09-05), per the "per-repo now, ruleset as follow-up" decision.

The finding that prompted this

The infrastructure backlog design (and issue #54's outcome comment) stated that nightowlstudiollc/cleanroom was protected "because nightowlstudiollc has a Claude blocking review org ruleset." That is wrong. The ruleset exists but is enforcement: disabled:

$ gh api orgs/nightowlstudiollc/rulesets --jq '.[] | "\(.name)\t\(.enforcement)"'
Claude blocking review    disabled

$ gh api repos/nightowlstudiollc/cleanroom/rules/branches/main
[]

Every currently-protected repo in both orgs is protected by classic per-repo branch protection, not by any ruleset. The ruleset has been contributing nothing since it was created 2026-04-30. This was a label-matching error: a ruleset named "Claude blocking review" was credited without reading its enforcement field.

The work

Per-repo protection now covers 33 of 37 non-archived repos, but it does not scale: every new repo needs it applied by hand, and repo-template cannot carry branch protection in the template itself.

Evaluate whether to enable org-level rulesets as the durable mechanism:

  1. Decide what the disabled nightowlstudiollc ruleset was meant to do and whether to enable it or delete it. It targets ~ALL repos with a workflows rule pointing at nightowlstudiollc/.github's claude-required-review.yml. Note this is the workflows rule type, not required_status_checks — different mechanism from what per-repo protection uses, so it is not a like-for-like replacement.
  2. Test on one repo before any ~ALL rollout. A ruleset covering all repos hits repos with no review workflow (see [infra] Four repos have no enforced review check (needs a review workflow first) #89), which would block every PR there.
  3. Determine interaction with existing per-repo protection — rulesets and branch protection layer rather than replace, and the combined effect needs to be measured, not assumed.
  4. If adopted, create the equivalent on smartwatermelon (currently zero rulesets) and document which mechanism is authoritative so the next audit does not repeat this error.

Validate against a known-bad case. Confirm the ruleset actually blocks a PR that should fail before trusting it — a ruleset that reports as active while enforcing nothing is precisely the failure being corrected here.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions