Skip to content

Support allowed-branches enforcement for create-pull-request safe output#33610

Merged
dsyme merged 11 commits into
mainfrom
copilot/support-allowed-branches-create-pull-request
May 20, 2026
Merged

Support allowed-branches enforcement for create-pull-request safe output#33610
dsyme merged 11 commits into
mainfrom
copilot/support-allowed-branches-create-pull-request

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 20, 2026

This change adds branch-policy enforcement for safe-outputs.create-pull-request.allowed-branches: the create_pull_request collector now rejects payloads whose selected branch does not match configured branch globs. It also ensures the chosen payload branch remains the branch used in apply-time PR creation flow.

  • Problem scope

    • Add support for:
      safe-outputs:
        create-pull-request:
          allowed-branches: ["features/*"]
    • Enforce branch constraints at safe-output collection time, not only at apply time.
  • Collector/runtime enforcement (actions/setup/js)

    • Added allowed_branches parsing (array or comma-separated string) in safe_outputs_handlers.cjs.
    • Added glob-based branch matching for the effective entry.branch (including branch resolved from current checkout when agent input is empty/base).
    • Returns structured MCP error when branch does not match configured patterns.
  • Compiler/config plumbing (pkg/workflow)

    • Extended CreatePullRequestsConfig with allowed-branches.
    • Included allowed-branches in templatable expression-array preprocessing.
    • Emitted allowed_branches in handler config generation so runtime receives and enforces policy.
  • Type definitions

    • Updated JS config typings to include CreatePullRequestConfig["allowed-branches"].
  • Focused test coverage

    • Added collector tests for:
      • accepted resolved branch matching allowed_branches
      • rejected non-matching branch
    • Added parser/handler-config tests for:
      • expression parsing support for allowed-branches
      • allowed_branches emission in handler config
      • rejection of invalid bare-string input form (consistent with existing templatable list behavior)

Example

safe-outputs:
  create-pull-request:
    allowed-branches: ["feature/*", "release/*"]

If the agent emits:

{ "type": "create_pull_request", "branch": "hotfix/urgent", "title": "...", "body": "..." }

the collector returns an error and does not record the safe output, because hotfix/urgent does not match configured allowed-branches.

Copilot AI linked an issue May 20, 2026 that may be closed by this pull request
Co-authored-by: dsyme <7204669+dsyme@users.noreply.github.com>
Copilot AI changed the title [WIP] Add support for allowed-branches in create pull request Support allowed-branches enforcement for create-pull-request safe output May 20, 2026
Copilot finished work on behalf of dsyme May 20, 2026 18:44
Copilot AI requested a review from dsyme May 20, 2026 18:44
@dsyme dsyme marked this pull request as ready for review May 20, 2026 18:58
Copilot AI review requested due to automatic review settings May 20, 2026 18:58
Comment thread pkg/workflow/create_pull_request.go
@dsyme dsyme marked this pull request as draft May 20, 2026 19:02
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds support for enforcing a configured source-branch allowlist (safe-outputs.create-pull-request.allowed-branches) during create_pull_request safe-output collection, and plumbs the configuration through the workflow compiler into the JS runtime handler config.

Changes:

  • Extend CreatePullRequestsConfig and compiler handler-config generation to include allowed-branchesallowed_branches.
  • Enforce allowed_branches in the createPullRequestHandler, including when the branch is resolved from the current checkout.
  • Add/extend Go and JS tests to cover expression parsing/config emission and collector-time accept/reject behavior.
Show a summary per file
File Description
pkg/workflow/create_pull_request.go Adds AllowedBranches and treats allowed-branches as a templatable expression-array field.
pkg/workflow/config_parsing_helpers_test.go Updates tests to ensure allowed-branches supports expression parsing and rejects invalid bare strings.
pkg/workflow/compiler_safe_outputs_handlers.go Emits allowed_branches into the runtime handler config for create_pull_request.
pkg/workflow/compiler_safe_outputs_config_test.go Verifies allowed_branches is included/omitted in generated handler config JSON as expected.
actions/setup/js/types/safe-outputs-config.d.ts Adds CreatePullRequestConfig["allowed-branches"] typing.
actions/setup/js/safe_outputs_handlers.test.cjs Adds collector-time tests for allowed-branch enforcement (resolved branch acceptance + rejection).
actions/setup/js/safe_outputs_handlers.cjs Implements parsing and glob matching for allowed_branches and rejects non-matching branches during collection.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 6/7 changed files
  • Comments generated: 1

Comment thread actions/setup/js/safe_outputs_handlers.cjs
Co-authored-by: dsyme <7204669+dsyme@users.noreply.github.com>
Co-authored-by: dsyme <7204669+dsyme@users.noreply.github.com>
Copilot finished work on behalf of dsyme May 20, 2026 19:36
@dsyme dsyme marked this pull request as ready for review May 20, 2026 20:08
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 20, 2026

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 20, 2026

Design Decision Gate 🏗️ completed the design decision gate check.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 20, 2026

PR Code Quality Reviewer completed the code quality review.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 20, 2026

🧪 Test Quality Sentinel completed test quality analysis.

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Quality Review

I've reviewed the changes for allowed-branches enforcement in create-pull-request safe output. The implementation is solid overall with good test coverage and consistent patterns, but there's one critical inconsistency that needs to be fixed.

Critical Issue

Wildcard handling inconsistency — The new isAllowedBranch() function doesn't handle the literal "*" pattern as a true allow-all, which is inconsistent with existing policy checks in this codebase (isBaseBranchAllowed() in create_pull_request.cjs and similar functions in repo_helpers.cjs). This means allowed_branches: ["*"] would reject common branch names like feature/foo.

What Was Done Well

Comprehensive test coverage — Tests for both accepted and rejected branches
Schema updates — Proper updates to main_workflow_schema.json matching allowed-base-branches style
Documentation — Clear docs in safe-outputs.md and reference docs
Type safety — Updated TypeScript definitions
Consistent error messages — Clear, actionable error when branch doesn't match patterns
Expression support — Properly handles both array and expression forms

Recommendation

Add the pattern === "*" fast-path to isAllowedBranch() to match the existing codebase pattern for wildcard handling. This is a small fix that maintains consistency across all policy enforcement functions.

🔎 Code quality review by PR Code Quality Reviewer · ● 4.7M

Comment thread actions/setup/js/safe_outputs_handlers.cjs
Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /tdd and /grill-with-docs based on this being a security feature addition with comprehensive test coverage and documentation updates.

Key Themes

Test Coverage Gaps (/tdd)

  • Missing edge case tests for empty allowed_branches array vs undefined
  • Glob pattern behavior (especially ** vs *) not explicitly tested
  • The core logic is well-tested, but boundary conditions deserve explicit coverage

Terminology Consistency (/grill-with-docs)

  • Error messages and tool descriptions have minor inconsistencies between kebab-case (user-facing YAML) and snake_case (internal implementation)
  • Tool description doesn't fully reflect that branch validation happens after detection/fallback resolution
  • Struct field comments could use more parallel phrasing with related fields

Positive Highlights

Excellent test structure — the two new test cases cleanly demonstrate the happy path (matching pattern) and error path (non-matching pattern)

Consistent architecture — the implementation follows the existing pattern for allowed-base-branches, making the codebase more uniform

Complete plumbing — compiler, schema, handlers, docs all updated together (no missing pieces)

Security-first design — enforcement happens at collection time, not just apply time, which is the right defensive layer

Verdict

Commenting with suggested improvements. The additions are well-structured and secure, but the edge case test coverage and terminology alignment would strengthen the implementation before merge.


Resources:

  • /tdd skill: Focus on red-green-refactor, edge cases, and regression protection
  • /grill-with-docs skill: Ensure domain language consistency and clear user-facing terminology

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · ● 6.8M

Comments that could not be inline-anchored

actions/setup/js/safe_outputs_handlers.cjs:67

[/tdd] The isAllowedBranch function has a subtle edge case that isn't covered by tests: what happens when allowedPatterns is an empty array but still truthy?

Consider adding a test case:

it(&quot;should allow any branch when allowed_branches is empty array&quot;, async () =&gt; {
  handlers = createHandlers(mockServer, mockAppendSafeOutput, {
    create_pull_request: {
      allow_empty: true,
      allowed_branches: [], // explicitly empty
    },
  });

  const result = await handle…

</details>

<details><summary>actions/setup/js/safe_outputs_handlers.cjs:62</summary>

**[/tdd]** The glob matching logic uses `pathMode: true` for branch names, but branch names aren&#39;t filesystem paths. This works, but the test coverage doesn&#39;t verify the glob behavior with different patterns.

Consider adding tests for common glob patterns:

```javascript
it(&quot;should match branch with double-star glob pattern&quot;, async () =&gt; {
  handlers = createHandlers(mockServer, mockAppendSafeOutput, {
    create_pull_request: {
      allow_empty: true,
      allowed_branches: [&quot;feature/**&quot;], …

</details>

<details><summary>actions/setup/js/safe_outputs_handlers.cjs:89</summary>

**[/grill-with-docs]** The error message uses &quot;allowed-branches&quot; (kebab-case) but the internal config field is `allowed_branches` (snake_case). This inconsistency might confuse users.

Consider matching the user-facing YAML terminology:

```javascript
error: `Branch &#39;${entry.branch}&#39; does not match allowed-branches patterns. Allowed: ${allowedBranches.join(&quot;, &quot;)}`

This keeps the error message aligned with the frontmatter field name users write in their workflow config.

actions/setup/js/safe_outputs_tools.json:174

[/grill-with-docs] The updated description says branch name "MUST be given" when allowed-branches is configured, but looking at the handler code (line 81-95 in safe_outputs_handlers.cjs), it enforces the resolved branch including fallback.

The description should clarify:

&quot;Source branch name containing the changes. If omitted, uses the current working branch. When safe-outputs.create-pull-request.allowed-branches is configured, the effective branch (agent-provided or detected fro…

</details>

<details><summary>pkg/workflow/create_pull_request.go:415</summary>

**[/grill-with-docs]** The field comment uses slightly different terminology than `AllowedBaseBranches` above it. Compare:

- Line 414: &quot;Enables agent-provided `base` override when configured.&quot;
- Line 416: &quot;Branch in create_pull_request payload must match when configured.&quot;

Consider parallel phrasing:

```go
AllowedBranches []string `yaml:&quot;allowed-branches,omitempty&quot;` // List of allowed source branch globs (e.g. &quot;feature/*&quot;). Enforces agent-provided or detected `branch` when configured.

Th…

@github-actions
Copy link
Copy Markdown
Contributor

🏗️ Design Decision Gate — ADR Required

This PR adds ~187 new lines in business-logic directories (pkg/workflow/, actions/setup/js/) without a linked Architecture Decision Record (ADR). The change introduces a new public configuration surface (safe-outputs.create-pull-request.allowed-branches) and a new enforcement point at MCP safe-output collection time — both of which are decisions worth recording.

AI has analyzed the PR diff and generated a draft ADR to help you get started:

📄 Draft ADR: docs/adr/33610-allowed-branches-enforcement-for-create-pull-request.md

What to do next

  1. Review the draft ADR just committed to your branch — it was generated from the PR diff and the PR description.

  2. Complete the missing sections — at minimum, fill in the Deciders: field and confirm the alternatives reflect what you actually considered.

  3. Tighten the rationale if you weighed any constraints the AI couldn't infer (security model, prior incidents, API stability commitments, etc.).

  4. Reference the ADR in this PR body by adding a line such as:

    ADR: ADR-33610: Collector-Time allowed-branches Enforcement for create-pull-request Safe Output

Once the ADR is linked from the PR body, this gate will re-run and verify the implementation matches the recorded decision.

Why ADRs Matter

"AI made me procrastinate on key design decisions. Because refactoring was cheap, I could always say 'I'll deal with this later.' Deferring decisions corroded my ability to think clearly."

ADRs create a searchable, permanent record of why the codebase looks the way it does. Future contributors (and your future self) will thank you for explaining why source-branch policy is enforced in the MCP collector rather than only at apply time.

📋 Michael Nygard ADR Format Reference

An ADR must contain these four sections to be considered complete:

  • Context — What is the problem? What forces are at play?
  • Decision — What did you decide? Why?
  • Alternatives Considered — What else could have been done?
  • Consequences — What are the trade-offs (positive and negative)?

All ADRs in this repo are stored in docs/adr/ and numbered by PR (e.g., 33610-...md for PR #33610).

🔒 This PR cannot merge until an ADR is linked in the PR body.

References:

🏗️ ADR gate enforced by Design Decision Gate 🏗️ · ● 6.6M ·

@github-actions
Copy link
Copy Markdown
Contributor

🧪 Test Quality Sentinel Report

Test Quality Score: 91/100

Excellent test quality

Metric Value
New/modified tests analyzed 7
✅ Design tests (behavioral contracts) 7 (100%)
⚠️ Implementation tests (low value) 0 (0%)
Tests with error/edge cases 5 (71%)
Duplicate test clusters 0
Test inflation detected No (ratios justified)
🚨 Coding-guideline violations 0

📋 Test Classification Details
Test File Classification Issues Detected
should enforce create_pull_request allowed_branches against resolved branch actions/setup/js/safe_outputs_handlers.test.cjs:9 ✅ Design None
should reject create_pull_request when branch does not match allowed_branches actions/setup/js/safe_outputs_handlers.test.cjs:43 ✅ Design None
TestCreatePullRequestBaseBranch (modified) pkg/workflow/compiler_safe_outputs_config_test.go:112 ✅ Design None
TestParsePullRequestsConfigExpressionFields (modified) pkg/workflow/config_parsing_helpers_test.go:147 ✅ Design None
TestHandlerConfigExpressionFields (modified) pkg/workflow/config_parsing_helpers_test.go:160 ✅ Design None
TestExpressionFieldsRejectedForInvalidStrings (modified) pkg/workflow/config_parsing_helpers_test.go:178 ✅ Design None
TestSafeOutputsToolsJSONInSync (new) pkg/workflow/safe_outputs_tools_test.go:211 ✅ Design None

Test Quality Highlights

JavaScript Tests (vitest):

  • ✅ Both new tests verify behavioral contracts: branch pattern matching and rejection
  • ✅ Comprehensive assertions (4-5 per test) covering success and error paths
  • ✅ Legitimate I/O mocking (GitHub API server, safe output handlers)
  • ✅ Tests cover both positive (branch matches pattern) and negative (branch rejected) cases

Go Tests:

  • ✅ All test additions/modifications validate compiler behavior end-to-end
  • ✅ Table-driven tests used effectively with clear test cases
  • ✅ All assertions include descriptive messages
  • ✅ Multi-layer validation: schema parsing → expression handling → config generation → runtime sync
  • ✅ New TestSafeOutputsToolsJSONInSync prevents drift between compiler and runtime JSON files

Test Inflation Analysis:

  • safe_outputs_handlers.test.cjs: 1.08:1 ratio ✅
  • Go test files: High ratios (10:1) are justified by schema-driven validation where minimal production code changes (2 lines) require comprehensive multi-layer testing

Language Support

Tests analyzed:

  • 🐹 Go (*_test.go): 5 test modifications/additions
  • 🟨 JavaScript (*.test.cjs): 2 new tests

Verdict

Check passed. 0% of tests are implementation tests (threshold: 30%). All tests verify behavioral contracts with excellent coverage.

Score Breakdown:

  • Behavioral Coverage: 40/40 (100% design tests)
  • Error/Edge Case Coverage: 21/30 (71% with edge cases)
  • Low Duplication: 20/20 (no duplicates)
  • Proportional Growth: 10/10 (justified ratios)

📖 Understanding Test Classifications

Design Tests (High Value) verify what the system does:

  • Assert on observable outputs, return values, or state changes
  • Cover error paths and boundary conditions
  • Would catch a behavioral regression if deleted
  • Remain valid even after internal refactoring

Implementation Tests (Low Value) verify how the system does it:

  • Assert on internal function calls (mocking internals)
  • Only test the happy path with typical inputs
  • Break during legitimate refactoring even when behavior is correct
  • Give false assurance: they pass even when the system is wrong

Goal: Shift toward tests that describe the system's behavioral contract — the promises it makes to its users and collaborators.

🧪 Test quality analysis by Test Quality Sentinel · ● 9.1M ·

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Test Quality Sentinel: 91/100. Test quality is excellent — 0% of tests are implementation tests (threshold: 30%). All 7 new/modified tests verify behavioral contracts with comprehensive coverage.

@github-actions
Copy link
Copy Markdown
Contributor

Progress update: checks are green, the branch is behind main, and a base update was attempted but could not be pushed here.

Generated by 👨‍🍳 PR Sous Chef · ● 2.5M ·

@dsyme dsyme merged commit 6af41c9 into main May 20, 2026
33 checks passed
@dsyme dsyme deleted the copilot/support-allowed-branches-create-pull-request branch May 20, 2026 21:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support allowed-branches in create-pull-request

3 participants