Skip to content

feat(create-pr): align with gh-aw create-pull-request implementation#155

Open
jamesadevine wants to merge 3 commits intomainfrom
feat/create-pr-gh-aw-parity
Open

feat(create-pr): align with gh-aw create-pull-request implementation#155
jamesadevine wants to merge 3 commits intomainfrom
feat/create-pr-gh-aw-parity

Conversation

@jamesadevine
Copy link
Copy Markdown
Collaborator

Summary

Closes the gap between ado-aw and gh-aw's create-pull-request safe output implementation. Identified via a systematic comparison of both codebases.

Security/Correctness

  • File protection system — blocks manifests (package.json, go.mod, Cargo.toml, etc.), CI configs (.github/, .pipelines/), and agent instruction files (.agents/, .claude/, .copilot/) by default. Override with protected-files: allowed.
  • Max files per PR — caps at 100 files (configurable via max-files)
  • Patch conflict fallbackgit am --3way with git apply --3way fallback instead of hard failure
  • Branch salt randomness — cryptographic random hex via rand crate instead of predictable timestamp

Feature Parity

  • Draft PR support — default draft: true, operator-enforced via ADO isDraft field
  • Fallback-as-work-item — records branch info on PR creation failure for manual recovery
  • Excluded filesexcluded-files: glob patterns strip files from patches before application
  • No-changes configif-no-changes: warn|error|ignore controls behavior on empty patches
  • Label validation — agent can pass labels in tool params, validated against allowed-labels allowlist. Operator labels always applied unconditionally.
  • Title prefixtitle-prefix: "[Bot] " prepended to all PR titles

Patch Format Upgrade

  • Migrated from raw git diff to git format-patch for proper commit metadata, rename detection, and binary file handling
  • Stage 2 applies patches via git am --3way with git apply --3way fallback
  • Added git to default bash command allowlist so agents can commit
  • Updated tool description to encourage staging commits before PR creation

Other

  • Provenance footer appended to PR body with timestamp and compiler version
  • ExecutionResult::failure_with_data for structured error responses

New Config Fields

safe-outputs:
  create-pull-request:
    draft: true                    # default: true (NEW)
    title-prefix: "[Bot] "        # NEW
    if-no-changes: warn           # warn|error|ignore (NEW)
    max-files: 100                # NEW
    protected-files: blocked      # blocked|allowed (NEW)
    excluded-files: ["*.lock"]    # NEW
    allowed-labels: ["automated"] # NEW
    fallback-as-work-item: true   # default: true (NEW)

Testing

All 651 tests pass (599 unit + 35 compiler + 8 mcp-http + 9 proxy).

Closes the gap between ado-aw and gh-aw's create-pull-request safe output:

Security/Correctness:
- Add file protection system blocking manifests (package.json, go.mod,
  Cargo.toml, etc.), CI configs (.github/, .pipelines/), and agent
  instruction files (.agents/, .claude/, .copilot/) by default
- Add max-files limit per PR (default: 100, configurable)
- Add 3-way merge fallback when patch application fails on conflicts
- Replace predictable timestamp-based branch suffix with cryptographic
  random hex (rand crate)

Feature Parity:
- Add draft PR support (default: true, operator-enforced via isDraft)
- Add fallback-as-work-item: record branch info on PR creation failure
- Add excluded-files glob config to filter files from patches
- Add if-no-changes config (warn/error/ignore) for empty patches
- Add allowed-labels allowlist to restrict agent-provided labels
- Add title-prefix config for operator branding
- Add agent-provided labels parameter validated against allowed-labels

Patch Format:
- Migrate from raw git diff to git format-patch for proper commit
  metadata, rename detection, and binary file handling
- Stage 2 applies patches via git am --3way with git apply fallback
- Add collect_changes_from_diff_tree for committed patch changes
- Add git to default bash command allowlist so agents can commit
- Update tool description to encourage staging commits before PR creation

Other:
- Add provenance footer to PR body with timestamp and compiler version
- Add ExecutionResult::failure_with_data for structured error responses

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jamesadevine jamesadevine force-pushed the feat/create-pr-gh-aw-parity branch from 0b26f4d to 3caf5d7 Compare April 11, 2026 16:18
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Rust PR Review

Summary: Mostly solid — the feature parity work is well-structured and the security additions are a real improvement — but there are two correctness bugs and an edge case worth addressing before merge.

Findings

🐛 Bugs / Logic Issues

  • src/tools/create_pr.rsif-no-changes: "warn" is identical to "error"
    Both the excluded-files path (~line 594) and the no-changes path (~line 828) use a match where _ (the "warn" catch-all) returns ExecutionResult::failure() — the same as "error". Operators who configure if-no-changes: warn will still see a pipeline failure, which defeats the purpose of a distinct "warn" mode. The _ arm should return ExecutionResult::success() (mirroring "ignore") while logging a warning. Right now the three-variant enum exposed in the docs/config effectively has only two behaviors.

  • src/mcp.rs — Silently-ignored git reset HEAD~1 is a correctness risk
    The temporary commit reset is fire-and-forget:

    let _ = Command::new("git")
        .args(["reset", "HEAD~1", "--mixed", "--quiet"])
        .current_dir(&git_dir)
        .output()
        .await;

    If this fails (e.g. permissions, detached HEAD, or filesystem error), the agent's repository silently retains the synthetic "agent changes" commit. Any subsequent git operations in the same pipeline run — including a second create-pull-request call — will operate on top of an unexpected extra commit. At minimum this should propagate an error; ideally, the failure is surfaced rather than swallowed.

⚠️ Suggestions

  • src/tools/create_pr.rsextract_paths_from_patch / filter_excluded_files_from_patch break on paths with spaces
    Both functions use split_whitespace() to parse diff --git a/path b/path headers. Git format-patch quotes space-containing filenames in the diff header (e.g. diff --git "a/path with spaces" "b/path with spaces") — splitting by whitespace will produce garbage paths for those cases. This means max-files counting and excluded-files filtering silently misbehave for repos with spaces in filenames. A more robust approach is to parse the --- a/ / +++ b/ lines instead, or use a dedicated diff-header parser.

  • src/tools/create_pr.rsfind_disallowed_files is #[cfg(test)] dead code in production
    The function lives under #[cfg(test)], but the tests use it with file path globs (not labels), hinting at a planned "allowed-files" feature that was never wired into the production executor. If there's no follow-up intent, this should be removed to avoid confusion; if there is, a tracking comment would help.

✅ What Looks Good

  • The switch from git diff to git format-patch + git am --3way with git apply --3way fallback is a meaningful robustness upgrade and handles the rename/binary file cases cleanly.
  • PROTECTED_MANIFEST_BASENAMES / PROTECTED_PATH_PREFIXES are thoughtfully comprehensive and the case-insensitive matching is correct.
  • Cryptographic branch salt via rand::rng().random::<u32>() is a clear improvement over the millisecond-timestamp approach.
  • The label allowlist validation (merge operator labels + validate agent labels) is implemented correctly in production — inline config.allowed_labels.contains(l) does the right thing.
  • collect_changes_from_diff_tree correctly handles the M/A/D/R diff-tree status codes and calls validate_single_path for all paths including both sides of a rename.

Generated by Rust PR Reviewer for issue #155 · ● 882.4K ·

- Fix if-no-changes 'warn' mode: add ExecutionResult::warning() with
  dedicated exit code 2. Pipeline step maps exit code 2 to
  ##vso[task.complete result=SucceededWithIssues;] for yellow badge.
  Previously 'warn' was identical to 'error' (both returned failure).

- Propagate git reset HEAD~1 failure in generate_patch instead of
  silently swallowing it. A failed reset would leave a synthetic commit
  in the repo, breaking subsequent operations.

- Fix path extraction for filenames with spaces: switch
  extract_paths_from_patch and filter_excluded_files_from_patch from
  parsing 'diff --git' headers (breaks on quoted paths) to using
  '--- a/' and '+++ b/' lines which handle spaces correctly.

- Remove dead find_disallowed_files function (was cfg(test) only,
  never wired into production executor).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Rust PR Review

Summary: Needs changes — good security additions, but the patch generation approach has a fundamental design conflict between the implementation and the updated tool description, plus binary file support is incomplete despite being claimed.

Findings

🐛 Bugs / Logic Issues

  • src/mcp.rs:194-290 — Format-patch captures uncommitted state only, but tool description says to commit first (critical)

    The synthetic-commit approach (git add -A && git commit && format-patch -1 && reset HEAD~1) only captures working-tree and staged changes — it cannot see previously committed work. Yet the updated create-pull-request tool description explicitly instructs agents:

    "stage and commit your changes using git add and git commit — each logical change should be its own commit"

    If an agent follows this advice and has committed their changes, the working tree is clean. git add -A stages nothing, git commit --allow-empty produces an empty commit, and git format-patch -1 yields a patch with no file changes. The result is an empty patch → if-no-changes: warn fires, and no PR is created. The previous git diff origin/main approach captured committed changes correctly.

    Either the implementation needs to use git format-patch origin/main..HEAD (or equivalent) to capture real commits, or the tool description must be reverted to "do not commit before calling this tool."

  • src/tools/create_pr.rs:1100-1175 — Binary files silently fail despite "binary file handling" being listed as a feature

    collect_changes_from_diff_tree reads file content with tokio::fs::read_to_string, which returns an Err for non-UTF-8 files. The ADO Pushes API supports contentType: "base64encoded" for binary files, but this code always sends contentType: "rawtext". Binary file support exists in Stage 1 (format-patch captures them) but is broken in Stage 2.

  • src/tools/create_pr.rs:1155-1175 — Renamed+modified files lose content changes

    When collect_changes_from_diff_tree encounters an R (rename) status entry, it emits only a rename change without reading the new file's content. If the file was simultaneously modified (partial rename similarity score, e.g., R80), the content delta is silently dropped. The ADO rename changetype only moves the path — it doesn't apply content edits.

  • src/main.rs:243 — Fragile success_count - warning_count arithmetic

    success_count - warning_count  // usize subtraction

    This assumes the invariant warning == true → success == true, which holds for all current constructors but is not enforced by the type system. A future ExecutionResult { success: false, warning: true, .. } — e.g., in failure_with_data — would cause a debug-mode panic. At minimum, add a debug_assert!(warning_count <= success_count) to make the assumption visible, or count with filter(|r| r.success && !r.warning) directly (as done in execute.rs, but main.rs uses the subtraction form).

⚠️ Suggestions

  • src/tools/create_pr.rsexcluded-files glob patterns don't match nested paths

    glob_match::glob_match("*.lock", "subdir/Cargo.lock") returns false* in the glob-match crate does not cross path separators. A user writing excluded-files: ["*.lock"] expecting to exclude all lockfiles will be surprised when services/api/package-lock.json slips through. Either document this prominently in the config doccomment, or normalise patterns (prepend **/ when the pattern contains no /) to match user intent.

  • src/tools/create_pr.rs:382,388 — Unvalidated string enums

    Both if_no_changes and protected_files are free-form String fields with no validation. A typo like protected-files: "disbaled" silently enables file protection (fail-safe, but silent). Consider adding a compile-time validation step in CreatePrResult::execute that rejects unrecognised values with a clear error, or change these to proper enums.

  • src/mcp.rs:206-222git commit may fail if git identity is not configured

    The synthetic commit requires user.email and user.name to be set in git config. In a fresh ADO agent, these may not be configured, causing git commit to fail with a "Please tell me who you are" error. Consider using -c user.email=agent@ado-aw -c user.name="ADO Agent" flags on the commit invocation (or the equivalent GIT_AUTHOR_*/GIT_COMMITTER_* env vars) to ensure the step is self-contained.

✅ What Looks Good

  • Cryptographic random suffix (rand::rng().random()) replacing the timestamp-based one is a solid improvement.
  • The filter_excluded_files_from_patch patch-block parser correctly handles the format-patch preamble headers (lines before the first diff --git) and quoted paths with spaces.
  • extract_paths_from_patch correctly excludes /dev/null from new-file entries.
  • The warning / ExecutionResult::warning / exit-code-2 / ##vso[task.complete result=SucceededWithIssues] chain is a clean end-to-end implementation.
  • Protected-files list and path-prefix detection are comprehensive and case-insensitive.
  • Label allowlist validation (allow-all when allowed_labels is empty, reject-unknown otherwise) is a sensible operator-friendly default.

Generated by Rust PR Reviewer for issue #155 · ● 1.6M ·

… globs, validation

- Fix format-patch to capture both committed and uncommitted changes
  by diffing against merge-base instead of only the last synthetic commit.
  Previously, if the agent committed changes per the tool description,
  the working tree was clean and format-patch produced an empty patch.

- Add git identity flags (-c user.email/user.name) to synthetic commit
  to avoid "Please tell me who you are" errors in fresh environments.

- Handle binary files in Stage 2 by falling back to base64 encoding
  when read_to_string fails (non-UTF-8 content). Uses ADO API
  contentType: "base64encoded" for binary files.

- Fix renamed+modified files losing content changes. When diff-tree
  reports R<score> with score < 100, emit both a rename and an edit
  change entry with the new file content.

- Fix fragile success_count - warning_count usize subtraction in
  execution summary. Count success as success && !warning directly
  instead of subtracting, preventing potential underflow.

- Normalize excluded-files glob patterns by auto-prepending **/ to
  patterns without /, so *.lock matches subdir/Cargo.lock.

- Validate if-no-changes and protected-files string values at
  execution time, rejecting typos with a clear error message.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Rust PR Review

Summary: Solid implementation with good security thinking, but has one correctness bug in the merge-base fallback, a misleading feature name, and a couple of edge cases worth addressing.

Findings

🐛 Bugs / Logic Issues

  • src/mcp.rsfind_merge_base fallback breaks on single-commit repos
    When neither origin/HEAD nor origin/main is reachable, the function returns the literal string "HEAD~1". On a repo with only one commit (e.g. fresh git init + one commit), HEAD~1 doesn't exist, so git format-patch HEAD~1..HEAD fails with fatal: bad revision 'HEAD~1..HEAD'. The error surface at that point is an opaque format-patch failure rather than a useful message. Fix: resolve the initial commit instead:

    // Fallback: diff from root commit
    let root = Command::new("git")
        .args(["rev-list", "--max-parents=0", "HEAD"])
        .current_dir(git_dir).output().await.ok();
    if let Some(out) = root.filter(|o| o.status.success()) {
        let sha = String::from_utf8_lossy(&out.stdout).trim().to_string();
        if !sha.is_empty() { return Ok(sha); }
    }
    // True fallback if even root fails (empty repo)
    return Err(anyhow_to_mcp_error(anyhow::anyhow!("Cannot determine diff base: no commits or remote")));
  • src/tools/create_pr.rs:extract_paths_from_patch — renames counted twice toward max_files
    Both --- a/<old> and +++ b/<new> lines are collected, so a rename counts as 2 entries and inflates the file count check. For the max_files budget only the destination path (+++ b/) should be counted; source paths are redundant for adds/edits too (where --- a/ and +++ b/ are the same file). The protected-files check is unaffected since it guards both old and new path names.

  • src/tools/create_pr.rs:filter_excluded_files_from_patch — empty-commit envelopes in multi-commit format-patch output
    When all diffs within a single commit block are excluded, the From <SHA> Mon Sep 17 ... / Subject: envelope remains in the output without any diff hunks. git am accepts this as an empty commit, which is mostly harmless but could leave unexpected empty commits on the branch. Worth noting in a comment since it's a non-obvious behaviour change from the old git diff path.

⚠️ Suggestions

  • fallback_as_work_item name is misleading
    The config key and PR description both imply a work item is automatically created on PR failure. In reality, ExecutionResult::failure_with_data just surfaces structured JSON in the failure log — no ADO work-item API is called. Operators who enable this expecting an auto-created work item will be surprised. Consider renaming to record-branch-on-failure (or documenting explicitly that it records branch info only) before this ships.

  • if_no_changes and protected_files validated only at Stage 2
    Both fields are String types with runtime enum validation. Invalid values (protected-files: bloced) only surface at execution time, not at ado-aw compile. Modelling them as Rust enums with #[serde(rename_all = "kebab-case")] would give pipeline authors compile-time feedback instead. This is consistent with how other strongly-typed fields in types.rs work.

  • src/tools/create_pr.rscollect_changes_from_worktree "RM" handling emits both a rename + edit
    The new code adds a second edit change entry for the new path when status is "RM" (renamed + modified in working tree). Confirm the ADO pushes API correctly handles a rename and a subsequent edit for the same new path in a single commit payload; otherwise the push may fail with a duplicate-path error.

✅ What Looks Good

  • Cryptographic random for branch IDs — upgrading from a predictable timestamp suffix to rand::rng().random::<u32>() is a correct security improvement.
  • ##vso[task.complete result=SucceededWithIssues] pattern in base.yml — the exit-code 2 convention piped through the ADO task result marker is exactly right for surfacing warnings without failing the pipeline.
  • Synthetic-commit cleanup ordering — the git reset HEAD~1 --mixed runs before checking format_patch_output.status, so the worktree is always restored even when format-patch fails. Clean.
  • Protected-files list — comprehensive, covers supply-chain attack vectors (manifests), CI configs, and agent instruction directories. Good defaults.
  • Binary file support via base64read_file_change now UTF-8-probes and falls back to base64encoded, which is the correct ADO push API content type for binary blobs.

Generated by Rust PR Reviewer for issue #155 · ● 984.1K ·

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant