Skip to content

ci: judge the pull-request description against the disclosure policy - #413

Merged
transitrix merged 1 commit into
mainfrom
ci/pr-description-policy
Jul 30, 2026
Merged

ci: judge the pull-request description against the disclosure policy#413
transitrix merged 1 commit into
mainfrom
ci/pr-description-policy

Conversation

@vkgeorgia

@vkgeorgia vkgeorgia commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

Adds a check that judges the pull-request title and body against this repository's disclosure policy — the part of it that is not a pattern.

public-surface-hygiene matches strings. Whether a description explains why a change was wanted, or recounts how a rule came about instead of stating what changed and how it works, is a judgment rather than a match. This step asks a model over GitHub Models using the built-in token, so no key and no external service are involved.

Three properties, all deliberate:

  • It prints nothing it flagged. The verdict is a category and one sentence written not to quote the text, and the model's raw output is never echoed. A gate that repeats what it caught becomes the channel it was meant to close.
  • It fails open on infrastructure, closed on a verdict. An unreachable, rate-limited or unavailable endpoint — a fork pull request has no models access — warns and passes; the pattern-based checks are unaffected, so nobody is pushed into deleting the check to merge.
  • policy:reviewed skips it. A model's judgment must not be able to hold a merge hostage. Deterministic rules have no such hatch and should not.

The rubric is public and every example in it is invented.

Test plan

Run locally against the live endpoint and against a stub:

  • four descriptions that should be rejected, one per policy category → rejected, category reported, flagged text not printed
  • seven that should pass, including real merged descriptions from this repository and its sibling → passed
  • verdict true → exit 1; verdict false → exit 0
  • HTTP 500, unreachable endpoint, unreadable verdict, absent token, empty description → warn and pass
  • node --check on the script; workflow parses as YAML

The pattern-based checks match strings. Whether a description explains why the
change was wanted, or recounts how a rule came about rather than stating what
changed and how it works, is a judgment. This step asks a model over GitHub
Models with the built-in token - no key, no external service - and fails the
check on a clear verdict.

It prints no part of the text it flagged, fails open on infrastructure and
closed on a verdict, and a pull request labelled policy:reviewed skips it: a
model's judgment must not be able to hold a merge hostage. Deterministic rules
have no such hatch.

Rubric and examples are public and every example in it is invented.
@transitrix
transitrix merged commit c0a391c into main Jul 30, 2026
9 of 10 checks passed
@transitrix
transitrix deleted the ci/pr-description-policy branch July 30, 2026 11:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants