Pattern quality: Emit tools.github.min-integrity for PR-review archetypes - #254
Merged
Merged
Conversation
The pr-review and skill-pr-reviewer archetypes' own tips already recommend setting tools.github.min-integrity so the agent only acts on content from trusted authors, but the generator only surfaced this as a prose bullet in the requirements list rather than baking it into the emitted workflow YAML. A downstream agent following the suggested draft was therefore likely to omit this guardrail entirely, leaving PR-triggered review workflows exposed to untrusted external-contributor content. Add a min_integrity field to these two archetype definitions in patterns/workflow-generation.json and have generateWorkflowFile emit tools.github.min-integrity in the generated frontmatter whenever it is set, so the safety-critical setting ships by default instead of relying on prose-only guidance. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
Author
There was a problem hiding this comment.
Reviewer: matt-pocock (JS/generator logic change)
Reviewed the diff for src/js/workflow.js, patterns/workflow-generation.json, and test/workflow.test.js.
The change is small, well-scoped, and safe:
min-integrityis only emitted inside the existinginferred.githubToolsetsbranch, so it can't leak into workflows without atools.githubblock.- Guard is a simple truthy check (
if (minIntegrity)), consistent with the surrounding code style (though it silently accepts any string value — fine givenmin_integrityis only ever set to"approved"in the pattern data today). - New tests cover both the positive case (
pr-reviewgetsmin-integrity: approved) and the negative case (status-reportdoesn't emit it at all), which is good regression coverage for this data-driven generator.
No blocking issues found. Approving is not requested per dispatcher rules, so leaving this as a COMMENT review — no changes needed before merge (already merged).
Generated by Specialist PR Review for #254 · copilot · auto · 18.9 AIC · ⌖ 5.38 AIC · ⊞ 8.1K
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was evaluated
Reviewed all 6 samples in
/tmp/gh-aw/data/generated-patterns-and-prompts.json(status-report, issue-triage, code-improvement, documentation-updater, dependency-monitor, pr-review) againstpatterns/manifest.json,patterns/workflow-generation.json,patterns/archetypes/*.json, and the generator logic insrc/js/workflow.js.Recurring quality issue found
For the
pr-review-readysample (and the closely relatedskill-pr-reviewerarchetype), the archetype's own curated tip already says:This tip is data the wizard already knows is safety-critical for
pull_request-triggered workflows, since PR content can come from untrusted external contributors. However, the generator only surfaced it as a prose "for example" bullet in the requirements list — it was never baked into the emittedtools.github:block of the suggested workflow YAML draft. A downstream agent implementing the draft is likely to treat prose tips as optional guidance and ship a workflow that reviews/reacts to unverified external PR content without the integrity gate, undermining the "Include DO NOT constraints to avoid false positive comments" goal stated elsewhere in the same prompt.What changed
patterns/workflow-generation.json: added"min_integrity": "approved"to thepr-reviewandskill-pr-reviewerarchetype definitions (both arepull_request/ready_for_review-triggered reviewers that read potentially untrusted diff/PR content).src/js/workflow.js:generateWorkflowFilenow emitstools.github.min-integrity: <value>directly in the generated frontmatter whenever an archetype definesmin_integrity, so the guardrail ships by default in the suggested draft instead of only appearing as prose.test/workflow.test.js: added assertions that pr-review workflows includemin-integrity: approvedundertools.github, and that unrelated archetypes (e.g.status-report) do not emitmin-integrityat all.Validation
npm test: 319/319 tests pass (18 test files).npm run build: production build succeeds,dist/patterns/archetypes/pr-review.jsonregenerated correctly.Follow-ups deliberately left out
min_integrityto other pull_request-related archetypes (e.g.pr-iteration-loop,accessibility-expert,user-simulator,community-digest,ci-failure-triage) since those don't primarily process untrusted external diff content the way PR reviewers do — a broader audit of which archetypes need this guardrail is worth a separate pass.patterns/from raw scan data, per task constraints.