Skip to content

feat(review-pr): add exclude-paths input to skip generated files from review#188

Merged
derekmisler merged 6 commits into
docker:mainfrom
docker-agent:fix/pr-review-exclude-paths
May 7, 2026
Merged

feat(review-pr): add exclude-paths input to skip generated files from review#188
derekmisler merged 6 commits into
docker:mainfrom
docker-agent:fix/pr-review-exclude-paths

Conversation

@docker-agent
Copy link
Copy Markdown
Contributor

Summary

Fixes a timeout issue in the PR review pipeline caused by generated protobuf files dominating the review queue.

Root cause: docker/agentic-platform PR #1965 (4338-line diff) timed out because session_service.pb.go was assigned risk score 6 (matched session in the security-path pattern + >100 added lines), landing in Chunk 1 with highest priority. The agent spent ~77k lines processing it with zero findings, leaving no capacity for real code.

Fix A — exclude-paths input

Adds optional exclude-paths input (newline-separated path prefixes) to both the reusable workflow and the composite action.

New step: 'Filter excluded paths from diff' runs after gh pr diff pre-fetch and before 'Split diff into chunks':

  • Uses pure bash+awk (no Python) to process diff section-by-section
  • Each section is buffered until the +++ b/<path> line is seen
  • If the path starts with any excluded prefix → the entire section is dropped
  • Logs each excluded file with ⏭️ Excluded from review: <path>
  • Logs summary: ✅ Filtered diff: N files excluded, M files remaining
  • Step only runs when pr.diff exists AND exclude-paths is non-empty

Fix B — Risk scorer ignores excluded paths + generated-file markers

Updated 'Score file risk' bash step:

  • Excluded paths check: files whose path starts with any exclude-paths prefix → score 0 (redundant safety net if the diff still has those sections for some reason)
  • Generated-file marker check: for files NOT in exclude-paths, inspects first 5 added lines for Code generated or DO NOT EDIT → score 0 (catches .pb.go and similar files not covered by a path prefix)

Files changed

  • .github/workflows/review-pr.yml: added exclude-paths to workflow_call inputs and threaded through to the review-pr action call
  • review-pr/action.yml: added exclude-paths input, new filter step, updated scoring step with EXCLUDE_PATHS env var and two new zero-score conditions

Consumer configuration

Downstream repos (e.g. docker/agentic-platform) can now configure:

uses: docker/cagent-action/.github/workflows/review-pr.yml@v...
with:
  exclude-paths: |
    backend/gen/
    frontend/src/gen/

Refs: analysis artifact 86ed46d9-3606-4072-b570-62204d354c86

@docker-agent docker-agent requested a review from a team as a code owner May 7, 2026 14:14
@docker-agent docker-agent requested a review from derekmisler May 7, 2026 14:14
Copy link
Copy Markdown
Contributor Author

@docker-agent docker-agent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assessment: 🟡 NEEDS ATTENTION

Two medium-severity bugs in the awk diff-filter bypass excluded paths for deleted files and pure renames. Input threading and main modification-file filtering are correct.

Comment thread review-pr/action.yml Outdated
Comment thread review-pr/action.yml Outdated
Comment thread review-pr/action.yml Outdated
Comment thread review-pr/action.yml Outdated
docker-agent added a commit to docker-agent/cagent-action that referenced this pull request May 7, 2026
…and empty result

Four issues found in review of PR docker#188:

[MEDIUM] Deleted files in excluded paths bypassed the filter.
When a generated file is deleted, git emits '+++ /dev/null' instead of
'+++ b/<path>', so the /^\+\+\+ b\// pattern never matched and the section
leaked through. Fix: add a '--- a/<path>' pattern; for deletions this is the
only real path line in the section.

[MEDIUM] Pure renames (100% similarity) in excluded paths bypassed the filter.
A rename-only section has no '---' or '+++' lines at all — only
'diff --git', 'similarity index', 'rename from', and 'rename to'. The
filter had no way to detect the path. Fix: add a 'rename to <path>'
pattern to catch these sections.

Both fixes use a shared check_prefix() awk function guarded by '!skip'
so only the first path-bearing line per section logs to stderr (avoids
double-counting modifications that have both '--- a/' and '+++ b/').

[LOW] Empty diff (all files excluded) was not short-circuited.
After filtering, if REMAINING_COUNT == 0 the output file is 0 bytes but
hashFiles('pr.diff') still returns a non-empty hash, causing the chunker
and scorer to fire unnecessarily. Fix: remove pr.diff when REMAINING_COUNT
is 0 so hashFiles returns '' and downstream steps are skipped.

[LOW] Clarified comment on generated-file marker check in Score file risk.
The awk scan only matches '+' lines (added lines), so it only catches
brand-new generated files. For pre-existing generated files being modified,
the header is a context line (space-prefixed) and is invisible to the check.
Updated the comment to state this limitation clearly so future maintainers
don't rely on it for the modification case.
derekmisler
derekmisler previously approved these changes May 7, 2026
Copy link
Copy Markdown
Contributor Author

@docker-agent docker-agent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assessment: 🟡 NEEDS ATTENTION

One medium-severity issue: the Score file risk step can silently succeed when score-risk.js fails (two-command pattern without set -euo pipefail). Two low-severity issues: the extractFilePath fix lacks a regression test that demonstrates the behavioral difference, and score-risk/ is missing from the AGENTS.md layout.

The TypeScript rewrites are otherwise correct and well-tested: all 207 unit tests pass, all four diff section types (modification, deletion, rename, new file) are handled correctly, scoring rules faithfully replicate the original bash, and applyFilter resolves the prior-round empty-diff guard issue by deleting the file when all sections are excluded.

Comment thread review-pr/action.yml
done < /tmp/file_diff_stats.txt

rm -f /tmp/file_diff_stats.txt
node "$ACTION_PATH/dist/score-risk.js" pr.diff "$EXCLUDE_PATHS"
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MEDIUM] score-risk.js failure is silently masked by the trailing echo command

The Score file risk step has two commands with no set -euo pipefail:

node "$ACTION_PATH/dist/score-risk.js" pr.diff "$EXCLUDE_PATHS"
echo "✅ File risk scores: $(jq -c . /tmp/file_risk_scores.json)"

If score-risk.js exits non-zero (binary not found, parse error, write error), bash continues to the echo line. The echo exits 0 even if the embedded jq fails, so the step reports success while /tmp/file_risk_scores.json is absent or stale. The pr-review.yaml agent reads this file directly and will silently operate on wrong/missing risk data, potentially misrouting the review.

Compare: filter-diff is a single-command step — its node failure propagates correctly.

Fix: add set -euo pipefail before the node call, or restructure as a single node ... && echo ... chain:

set -euo pipefail
node "$ACTION_PATH/dist/score-risk.js" pr.diff "$EXCLUDE_PATHS"
echo "✅ File risk scores: $(jq -c . /tmp/file_risk_scores.json)"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 4bc6182. The Score file risk step now begins with set -euo pipefail:

run: |
  set -euo pipefail
  node "$ACTION_PATH/dist/score-risk.js" pr.diff "$EXCLUDE_PATHS"
  echo "✅ File risk scores: $(jq -c . /tmp/file_risk_scores.json)"

If score-risk.js exits non-zero the step now fails immediately rather than continuing to the echo and masking the error.

'src/auth.spec.tsx',
'test_auth.py',
'docs/README.md',
'.github/workflows/ci.yml',
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[LOW] The extractFilePath fix has no test that exercises the behavioral difference

The PR description calls out fixing a greedy-regex bug in path extraction:

  • Old bash: sub(/.*b\//, "", file) — greedy, strips up to last b/, mangling paths like .github/workflows/ci.yml into ci.yml
  • New TS: indexOf(' b/') — finds the first b/ separator, returning the correct full path

The existing rule-6 test covers .github/workflows/ci.yml and confirms score = 0, but the score is identically 0 under both the old and new extraction:

  • Old: extracts ci.yml → matches \.ya?ml$ → reset to 0
  • New: extracts .github/workflows/ci.yml → matches \.ya?ml$ → reset to 0

No test demonstrates a case where the fix actually changes the result. A behavioral-difference test would be:

// Path where old bash extracted a segment matching a security keyword,
// but the full correct path does NOT (or vice versa).
// e.g., "diff --git a/.github/workflows/token-refresh.yml b/.github/workflows/token-refresh.yml"
// Old bash extracts: "token-refresh.yml" — no security keyword, no rule 3
// New TS extracts:   ".github/workflows/token-refresh.yml" — no security keyword either
// (both give 0 due to .yml reset — but it's harder to construct a case that differs)
// The cleaner test: verify extractFilePath directly, or test a .github/workflows/auth.sh
// (no .yml reset, but old bash would wrongly extract "auth.sh" vs correct ".github/workflows/auth.sh")

This is not a correctness bug in the shipped code — the fix is right — but a gap in the test's ability to serve as a regression guard.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 4bc6182. Added a dedicated describe block in src/score-risk/__tests__/score-risk.test.ts:

describe('scoreFiles — extractFilePath: indexOf regression for paths containing b/', () => {
  it('path with security keyword before a b/ directory component scores 2, not 0', () => {
    // Old greedy regex: replace(/.*b\//, '') on
    //   "diff --git a/src/auth/b/helper.ts b/src/auth/b/helper.ts"
    // matches everything up to the last 'b/' giving 'helper.ts'.
    // 'helper.ts' has no security keyword → score 0.
    //
    // New indexOf(' b/'): strips "diff --git a/" prefix and splits at the
    // first ' b/' separator giving 'src/auth/b/helper.ts'.
    // 'src/auth/b/helper.ts' matches SECURITY_PATH_RE ('auth') → score 2.
    const diff = makeDiff('src/auth/b/helper.ts', ['+changed']);
    const scores = scoreFiles(diff, []);
    expect(scores['src/auth/b/helper.ts']).toBe(2);
  });
});

This demonstrates a concrete case where old and new extraction produce different scores (0 vs 2). 208/208 tests pass.

Comment thread AGENTS.md
│ ├── filter-diff/ # Strips excluded-path sections from a unified diff.
│ │ ├── index.ts # CLI entry → bundled to dist/filter-diff.js
│ │ ├── filter-diff.ts # Core filterDiff() pure function + applyFilter() I/O wrapper.
│ │ └── __tests__/
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[LOW] score-risk/ module missing from AGENTS.md repo layout

AGENTS.md was updated to add the filter-diff/ section (lines 42–45), but src/score-risk/ — the other new module introduced by this PR — is absent from the layout. The layout section is the canonical map of what lives in src/ for future contributors and agents reading this file.

The missing entry should appear after filter-diff/ (alphabetically between filter-diff/ and get-pr-meta/):

│   ├── score-risk/                  # Per-file risk scoring for the PR review pipeline.
│   │   ├── index.ts                 # CLI entry → bundled to dist/score-risk.js
│   │   ├── score-risk.ts            # Core scoreFiles() pure function.
│   │   └── __tests__/

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 06b6243 — added score-risk/ between filter-diff/ and get-pr-meta/ in the repo layout section of AGENTS.md.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already fixed in 06b6243 (the commit immediately before this round's changes). score-risk/ is now present between filter-diff/ and get-pr-meta/ in the repo layout section of AGENTS.md.

Copy link
Copy Markdown
Contributor Author

@docker-agent docker-agent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assessment: 🟢 APPROVE

All round-2 findings resolved correctly in commit 4bc6182:

  • set -euo pipefail added at line 325 of review-pr/action.ymlscore-risk.js failure now propagates and fails the step correctly.
  • Regression test for extractFilePath fix added: src/auth/b/helper.ts path proves old greedy regex extracts helper.ts (score 0), new indexOf(' b/') extracts src/auth/b/helper.ts (score 2 via auth keyword). The behavioral difference is now demonstrable.
  • score-risk/ present in AGENTS.md layout at lines 46–49.

208/208 unit tests pass (verified locally on the PR branch).

… review

Fix A: Add `exclude-paths` input (newline-separated path prefixes) to:
- `.github/workflows/review-pr.yml`: reusable workflow inputs
- `review-pr/action.yml`: composite action inputs

Add 'Filter excluded paths from diff' step in review-pr/action.yml that
runs after diff pre-fetch and before chunking. Uses awk to process the diff
section-by-section; any section whose `+++ b/<path>` matches an excluded
prefix is stripped entirely. Logs each excluded file and a summary count.

Fix B: Update 'Score file risk' bash step to assign score 0 for:
- Files whose path starts with any `exclude-paths` prefix
- Files NOT in exclude-paths whose first 5 added lines contain
  'Code generated' or 'DO NOT EDIT' (catches generated files not
  covered by exclude-paths, e.g. .pb.go files)

This prevents generated protobuf files (e.g. backend/gen/**) from
landing in high-priority chunks and consuming agent budget before
the reviewer can process real code changes.

Root cause: docker/agentic-platform PR #1965 timed out because
session_service.pb.go scored 6 (security path + large change) and
consumed the entire first chunk, leaving no capacity for real code.
…and empty result

Four issues found in review of PR docker#188:

[MEDIUM] Deleted files in excluded paths bypassed the filter.
When a generated file is deleted, git emits '+++ /dev/null' instead of
'+++ b/<path>', so the /^\+\+\+ b\// pattern never matched and the section
leaked through. Fix: add a '--- a/<path>' pattern; for deletions this is the
only real path line in the section.

[MEDIUM] Pure renames (100% similarity) in excluded paths bypassed the filter.
A rename-only section has no '---' or '+++' lines at all — only
'diff --git', 'similarity index', 'rename from', and 'rename to'. The
filter had no way to detect the path. Fix: add a 'rename to <path>'
pattern to catch these sections.

Both fixes use a shared check_prefix() awk function guarded by '!skip'
so only the first path-bearing line per section logs to stderr (avoids
double-counting modifications that have both '--- a/' and '+++ b/').

[LOW] Empty diff (all files excluded) was not short-circuited.
After filtering, if REMAINING_COUNT == 0 the output file is 0 bytes but
hashFiles('pr.diff') still returns a non-empty hash, causing the chunker
and scorer to fire unnecessarily. Fix: remove pr.diff when REMAINING_COUNT
is 0 so hashFiles returns '' and downstream steps are skipped.

[LOW] Clarified comment on generated-file marker check in Score file risk.
The awk scan only matches '+' lines (added lines), so it only catches
brand-new generated files. For pre-existing generated files being modified,
the header is a context line (space-prefixed) and is invisible to the check.
Updated the comment to state this limitation clearly so future maintainers
don't rely on it for the modification case.
Implements the AGENTS.md convention that all new composite-action logic
must live in src/ as TypeScript with Vitest tests, not as inline bash
or awk in YAML.  Shell steps in action YAML should only orchestrate
calls to dist/*.js tools.

New module: src/filter-diff/
  filter-diff.ts   — Core logic split into two public exports:
    filterDiff(diffContent, excludePrefixes): FilterResult
      Pure function — no filesystem access.  Handles all diff section
      types correctly:
        Modifications  detected via '+++ b/<path>'
        Deletions      detected via '--- a/<path>' (+++ is /dev/null)
        Pure renames   detected via 'rename to <path>' (no ---/+++ lines)
      The !skip guard prevents double-logging on modifications (which
      have both '--- a/' and '+++ b/' pointing to the same path).
    applyFilter(diffPath, excludePathsStr): void
      I/O wrapper — reads the diff, calls filterDiff, writes result
      back in-place.  Deletes the file when remainingCount == 0 so
      hashFiles('pr.diff') returns '' and downstream GH Actions steps
      with 'if: hashFiles(pr.diff) != ''' are skipped cleanly.
  index.ts   — Thin CLI entry: node dist/filter-diff.js <diffPath> <excludePaths>

Tests: src/filter-diff/__tests__/filter-diff.test.ts (22 tests)
  Covers all diff section types, mixed diffs, all-excluded (file deletion),
  empty inputs, multiple prefixes, and I/O behaviour via temp files.
  150/150 total unit tests pass.

tsup.config.ts: added 'filter-diff': src('filter-diff') to entry map.
  Bundles to dist/filter-diff.js (3.38 KB, zero npm deps).

review-pr/action.yml: replace 96-line awk block with:
  node "$ACTION_PATH/dist/filter-diff.js" pr.diff "$EXCLUDE_PATHS"

AGENTS.md: document the TypeScript-first convention explicitly:
  'New logic in composite actions must be implemented as TypeScript in
  src/ with Vitest unit tests — not as inline bash, awk, or other
  scripting languages embedded in YAML files.'
  Added filter-diff to the repo layout section.
…th tests

Migrates the 88-line 'Score file risk' bash+awk block from review-pr/action.yml
to a testable, type-safe TypeScript module (src/score-risk/), following the
AGENTS.md convention established by the filter-diff migration.

New module: src/score-risk/
  score-risk.ts  Core logic with two public exports:
    scoreFiles(diffContent, excludePrefixes): RiskScores
      Pure function — no filesystem access.  Replicates the seven scoring
      rules exactly:
        Rule 1  exclude-paths prefix match  → score 0, skip remaining
        Rule 2  generated-file markers      → score 0, skip remaining
                (first 5 added lines contain 'Code generated' / 'DO NOT EDIT')
        Rule 3  security-sensitive path     → +2 (case-insensitive)
        Rule 4  large change (>100 lines)   → +2
        Rule 5  many hunks (>3 headers)     → +1
        Rule 6  test/doc/config file        → reset score to 0
        Rule 7  error-handling patterns     → +1 (case-sensitive)
      Rule ordering preserved from bash: rule 6 resets to 0, then rule 7
      still applies (e.g. a test file with error patterns scores 1).
    parseExcludePrefixes(str): string[]
      Shared helper — trims, strips CR, drops blank lines.

    extractFilePath improvement over bash awk:
      Uses 'diff --git a/'.length + indexOf(' b/') instead of greedy
      /.*b\//.  Fixes silent mis-extraction for paths containing 'b/'
      in a directory component (e.g. .github/workflows/ci.yml where
      'github/' contains 'b/').

  index.ts  CLI entry:
    node dist/score-risk.js <diffPath> <excludePaths>
    Reads diffPath, calls scoreFiles(), writes /tmp/file_risk_scores.json.

Tests: src/score-risk/__tests__/score-risk.test.ts (57 tests)
  Every scoring rule tested in isolation with boundary conditions,
  combined-rule scenarios (max score = 6), and edge cases.
  207/207 total unit tests pass.

tsup.config.ts: added 'score-risk': src('score-risk') to entry map.
  Bundles to dist/score-risk.js (3.42 KB, zero npm deps).

review-pr/action.yml: replace 88-line bash block with:
  node "$ACTION_PATH/dist/score-risk.js" pr.diff "$EXCLUDE_PATHS"
  echo "✅ File risk scores: $(jq -c . /tmp/file_risk_scores.json)"
  Step name, if: condition, and env: block are unchanged.
Adds the score-risk/ entry alphabetically between filter-diff/ and
get-pr-meta/, consistent with the filter-diff entry added in the
previous commit.
… test

[MEDIUM] Add set -euo pipefail to Score file risk step.
Without it, if score-risk.js exits non-zero bash continues to the echo
which exits 0, masking the failure and leaving /tmp/file_risk_scores.json
missing or stale. The pr-review agent reads this file directly, so a
silent failure here could misroute the entire review.

[LOW] Add behavioral regression test for extractFilePath greedy-regex fix.
The existing .github/workflows/ci.yml test produces score 0 under both
old and new extraction (both match \.ya?ml$), so it does not demonstrate
that the fix changes any outcome. The new test uses src/auth/b/helper.ts:
  Old replace(/.*b\//): extracts 'helper.ts' → no keyword → score 0
  New indexOf(' b/'):    extracts 'src/auth/b/helper.ts' → 'auth' → score 2
This proves the new implementation gives a different (correct) result for
paths containing a 'b/' directory component before a security keyword.
@docker-agent docker-agent force-pushed the fix/pr-review-exclude-paths branch from 4bc6182 to 690ad4e Compare May 7, 2026 15:41
@derekmisler derekmisler merged commit 9edbb76 into docker:main May 7, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants