feat(review-pr): add exclude-paths input to skip generated files from review#188
Conversation
docker-agent
left a comment
There was a problem hiding this comment.
Assessment: 🟡 NEEDS ATTENTION
Two medium-severity bugs in the awk diff-filter bypass excluded paths for deleted files and pure renames. Input threading and main modification-file filtering are correct.
…and empty result Four issues found in review of PR docker#188: [MEDIUM] Deleted files in excluded paths bypassed the filter. When a generated file is deleted, git emits '+++ /dev/null' instead of '+++ b/<path>', so the /^\+\+\+ b\// pattern never matched and the section leaked through. Fix: add a '--- a/<path>' pattern; for deletions this is the only real path line in the section. [MEDIUM] Pure renames (100% similarity) in excluded paths bypassed the filter. A rename-only section has no '---' or '+++' lines at all — only 'diff --git', 'similarity index', 'rename from', and 'rename to'. The filter had no way to detect the path. Fix: add a 'rename to <path>' pattern to catch these sections. Both fixes use a shared check_prefix() awk function guarded by '!skip' so only the first path-bearing line per section logs to stderr (avoids double-counting modifications that have both '--- a/' and '+++ b/'). [LOW] Empty diff (all files excluded) was not short-circuited. After filtering, if REMAINING_COUNT == 0 the output file is 0 bytes but hashFiles('pr.diff') still returns a non-empty hash, causing the chunker and scorer to fire unnecessarily. Fix: remove pr.diff when REMAINING_COUNT is 0 so hashFiles returns '' and downstream steps are skipped. [LOW] Clarified comment on generated-file marker check in Score file risk. The awk scan only matches '+' lines (added lines), so it only catches brand-new generated files. For pre-existing generated files being modified, the header is a context line (space-prefixed) and is invisible to the check. Updated the comment to state this limitation clearly so future maintainers don't rely on it for the modification case.
docker-agent
left a comment
There was a problem hiding this comment.
Assessment: 🟡 NEEDS ATTENTION
One medium-severity issue: the Score file risk step can silently succeed when score-risk.js fails (two-command pattern without set -euo pipefail). Two low-severity issues: the extractFilePath fix lacks a regression test that demonstrates the behavioral difference, and score-risk/ is missing from the AGENTS.md layout.
The TypeScript rewrites are otherwise correct and well-tested: all 207 unit tests pass, all four diff section types (modification, deletion, rename, new file) are handled correctly, scoring rules faithfully replicate the original bash, and applyFilter resolves the prior-round empty-diff guard issue by deleting the file when all sections are excluded.
| done < /tmp/file_diff_stats.txt | ||
|
|
||
| rm -f /tmp/file_diff_stats.txt | ||
| node "$ACTION_PATH/dist/score-risk.js" pr.diff "$EXCLUDE_PATHS" |
There was a problem hiding this comment.
[MEDIUM] score-risk.js failure is silently masked by the trailing echo command
The Score file risk step has two commands with no set -euo pipefail:
node "$ACTION_PATH/dist/score-risk.js" pr.diff "$EXCLUDE_PATHS"
echo "✅ File risk scores: $(jq -c . /tmp/file_risk_scores.json)"If score-risk.js exits non-zero (binary not found, parse error, write error), bash continues to the echo line. The echo exits 0 even if the embedded jq fails, so the step reports success while /tmp/file_risk_scores.json is absent or stale. The pr-review.yaml agent reads this file directly and will silently operate on wrong/missing risk data, potentially misrouting the review.
Compare: filter-diff is a single-command step — its node failure propagates correctly.
Fix: add set -euo pipefail before the node call, or restructure as a single node ... && echo ... chain:
set -euo pipefail
node "$ACTION_PATH/dist/score-risk.js" pr.diff "$EXCLUDE_PATHS"
echo "✅ File risk scores: $(jq -c . /tmp/file_risk_scores.json)"There was a problem hiding this comment.
Fixed in 4bc6182. The Score file risk step now begins with set -euo pipefail:
run: |
set -euo pipefail
node "$ACTION_PATH/dist/score-risk.js" pr.diff "$EXCLUDE_PATHS"
echo "✅ File risk scores: $(jq -c . /tmp/file_risk_scores.json)"If score-risk.js exits non-zero the step now fails immediately rather than continuing to the echo and masking the error.
| 'src/auth.spec.tsx', | ||
| 'test_auth.py', | ||
| 'docs/README.md', | ||
| '.github/workflows/ci.yml', |
There was a problem hiding this comment.
[LOW] The extractFilePath fix has no test that exercises the behavioral difference
The PR description calls out fixing a greedy-regex bug in path extraction:
- Old bash:
sub(/.*b\//, "", file)— greedy, strips up to lastb/, mangling paths like.github/workflows/ci.ymlintoci.yml - New TS:
indexOf(' b/')— finds the firstb/separator, returning the correct full path
The existing rule-6 test covers .github/workflows/ci.yml and confirms score = 0, but the score is identically 0 under both the old and new extraction:
- Old: extracts
ci.yml→ matches\.ya?ml$→ reset to 0 - New: extracts
.github/workflows/ci.yml→ matches\.ya?ml$→ reset to 0
No test demonstrates a case where the fix actually changes the result. A behavioral-difference test would be:
// Path where old bash extracted a segment matching a security keyword,
// but the full correct path does NOT (or vice versa).
// e.g., "diff --git a/.github/workflows/token-refresh.yml b/.github/workflows/token-refresh.yml"
// Old bash extracts: "token-refresh.yml" — no security keyword, no rule 3
// New TS extracts: ".github/workflows/token-refresh.yml" — no security keyword either
// (both give 0 due to .yml reset — but it's harder to construct a case that differs)
// The cleaner test: verify extractFilePath directly, or test a .github/workflows/auth.sh
// (no .yml reset, but old bash would wrongly extract "auth.sh" vs correct ".github/workflows/auth.sh")This is not a correctness bug in the shipped code — the fix is right — but a gap in the test's ability to serve as a regression guard.
There was a problem hiding this comment.
Fixed in 4bc6182. Added a dedicated describe block in src/score-risk/__tests__/score-risk.test.ts:
describe('scoreFiles — extractFilePath: indexOf regression for paths containing b/', () => {
it('path with security keyword before a b/ directory component scores 2, not 0', () => {
// Old greedy regex: replace(/.*b\//, '') on
// "diff --git a/src/auth/b/helper.ts b/src/auth/b/helper.ts"
// matches everything up to the last 'b/' giving 'helper.ts'.
// 'helper.ts' has no security keyword → score 0.
//
// New indexOf(' b/'): strips "diff --git a/" prefix and splits at the
// first ' b/' separator giving 'src/auth/b/helper.ts'.
// 'src/auth/b/helper.ts' matches SECURITY_PATH_RE ('auth') → score 2.
const diff = makeDiff('src/auth/b/helper.ts', ['+changed']);
const scores = scoreFiles(diff, []);
expect(scores['src/auth/b/helper.ts']).toBe(2);
});
});This demonstrates a concrete case where old and new extraction produce different scores (0 vs 2). 208/208 tests pass.
| │ ├── filter-diff/ # Strips excluded-path sections from a unified diff. | ||
| │ │ ├── index.ts # CLI entry → bundled to dist/filter-diff.js | ||
| │ │ ├── filter-diff.ts # Core filterDiff() pure function + applyFilter() I/O wrapper. | ||
| │ │ └── __tests__/ |
There was a problem hiding this comment.
[LOW] score-risk/ module missing from AGENTS.md repo layout
AGENTS.md was updated to add the filter-diff/ section (lines 42–45), but src/score-risk/ — the other new module introduced by this PR — is absent from the layout. The layout section is the canonical map of what lives in src/ for future contributors and agents reading this file.
The missing entry should appear after filter-diff/ (alphabetically between filter-diff/ and get-pr-meta/):
│ ├── score-risk/ # Per-file risk scoring for the PR review pipeline.
│ │ ├── index.ts # CLI entry → bundled to dist/score-risk.js
│ │ ├── score-risk.ts # Core scoreFiles() pure function.
│ │ └── __tests__/
There was a problem hiding this comment.
Fixed in 06b6243 — added score-risk/ between filter-diff/ and get-pr-meta/ in the repo layout section of AGENTS.md.
There was a problem hiding this comment.
Already fixed in 06b6243 (the commit immediately before this round's changes). score-risk/ is now present between filter-diff/ and get-pr-meta/ in the repo layout section of AGENTS.md.
docker-agent
left a comment
There was a problem hiding this comment.
Assessment: 🟢 APPROVE
All round-2 findings resolved correctly in commit 4bc6182:
set -euo pipefailadded at line 325 ofreview-pr/action.yml—score-risk.jsfailure now propagates and fails the step correctly.- Regression test for
extractFilePathfix added:src/auth/b/helper.tspath proves old greedy regex extractshelper.ts(score 0), newindexOf(' b/')extractssrc/auth/b/helper.ts(score 2 viaauthkeyword). The behavioral difference is now demonstrable. score-risk/present in AGENTS.md layout at lines 46–49.
208/208 unit tests pass (verified locally on the PR branch).
… review Fix A: Add `exclude-paths` input (newline-separated path prefixes) to: - `.github/workflows/review-pr.yml`: reusable workflow inputs - `review-pr/action.yml`: composite action inputs Add 'Filter excluded paths from diff' step in review-pr/action.yml that runs after diff pre-fetch and before chunking. Uses awk to process the diff section-by-section; any section whose `+++ b/<path>` matches an excluded prefix is stripped entirely. Logs each excluded file and a summary count. Fix B: Update 'Score file risk' bash step to assign score 0 for: - Files whose path starts with any `exclude-paths` prefix - Files NOT in exclude-paths whose first 5 added lines contain 'Code generated' or 'DO NOT EDIT' (catches generated files not covered by exclude-paths, e.g. .pb.go files) This prevents generated protobuf files (e.g. backend/gen/**) from landing in high-priority chunks and consuming agent budget before the reviewer can process real code changes. Root cause: docker/agentic-platform PR #1965 timed out because session_service.pb.go scored 6 (security path + large change) and consumed the entire first chunk, leaving no capacity for real code.
…and empty result Four issues found in review of PR docker#188: [MEDIUM] Deleted files in excluded paths bypassed the filter. When a generated file is deleted, git emits '+++ /dev/null' instead of '+++ b/<path>', so the /^\+\+\+ b\// pattern never matched and the section leaked through. Fix: add a '--- a/<path>' pattern; for deletions this is the only real path line in the section. [MEDIUM] Pure renames (100% similarity) in excluded paths bypassed the filter. A rename-only section has no '---' or '+++' lines at all — only 'diff --git', 'similarity index', 'rename from', and 'rename to'. The filter had no way to detect the path. Fix: add a 'rename to <path>' pattern to catch these sections. Both fixes use a shared check_prefix() awk function guarded by '!skip' so only the first path-bearing line per section logs to stderr (avoids double-counting modifications that have both '--- a/' and '+++ b/'). [LOW] Empty diff (all files excluded) was not short-circuited. After filtering, if REMAINING_COUNT == 0 the output file is 0 bytes but hashFiles('pr.diff') still returns a non-empty hash, causing the chunker and scorer to fire unnecessarily. Fix: remove pr.diff when REMAINING_COUNT is 0 so hashFiles returns '' and downstream steps are skipped. [LOW] Clarified comment on generated-file marker check in Score file risk. The awk scan only matches '+' lines (added lines), so it only catches brand-new generated files. For pre-existing generated files being modified, the header is a context line (space-prefixed) and is invisible to the check. Updated the comment to state this limitation clearly so future maintainers don't rely on it for the modification case.
Implements the AGENTS.md convention that all new composite-action logic
must live in src/ as TypeScript with Vitest tests, not as inline bash
or awk in YAML. Shell steps in action YAML should only orchestrate
calls to dist/*.js tools.
New module: src/filter-diff/
filter-diff.ts — Core logic split into two public exports:
filterDiff(diffContent, excludePrefixes): FilterResult
Pure function — no filesystem access. Handles all diff section
types correctly:
Modifications detected via '+++ b/<path>'
Deletions detected via '--- a/<path>' (+++ is /dev/null)
Pure renames detected via 'rename to <path>' (no ---/+++ lines)
The !skip guard prevents double-logging on modifications (which
have both '--- a/' and '+++ b/' pointing to the same path).
applyFilter(diffPath, excludePathsStr): void
I/O wrapper — reads the diff, calls filterDiff, writes result
back in-place. Deletes the file when remainingCount == 0 so
hashFiles('pr.diff') returns '' and downstream GH Actions steps
with 'if: hashFiles(pr.diff) != ''' are skipped cleanly.
index.ts — Thin CLI entry: node dist/filter-diff.js <diffPath> <excludePaths>
Tests: src/filter-diff/__tests__/filter-diff.test.ts (22 tests)
Covers all diff section types, mixed diffs, all-excluded (file deletion),
empty inputs, multiple prefixes, and I/O behaviour via temp files.
150/150 total unit tests pass.
tsup.config.ts: added 'filter-diff': src('filter-diff') to entry map.
Bundles to dist/filter-diff.js (3.38 KB, zero npm deps).
review-pr/action.yml: replace 96-line awk block with:
node "$ACTION_PATH/dist/filter-diff.js" pr.diff "$EXCLUDE_PATHS"
AGENTS.md: document the TypeScript-first convention explicitly:
'New logic in composite actions must be implemented as TypeScript in
src/ with Vitest unit tests — not as inline bash, awk, or other
scripting languages embedded in YAML files.'
Added filter-diff to the repo layout section.
…th tests
Migrates the 88-line 'Score file risk' bash+awk block from review-pr/action.yml
to a testable, type-safe TypeScript module (src/score-risk/), following the
AGENTS.md convention established by the filter-diff migration.
New module: src/score-risk/
score-risk.ts Core logic with two public exports:
scoreFiles(diffContent, excludePrefixes): RiskScores
Pure function — no filesystem access. Replicates the seven scoring
rules exactly:
Rule 1 exclude-paths prefix match → score 0, skip remaining
Rule 2 generated-file markers → score 0, skip remaining
(first 5 added lines contain 'Code generated' / 'DO NOT EDIT')
Rule 3 security-sensitive path → +2 (case-insensitive)
Rule 4 large change (>100 lines) → +2
Rule 5 many hunks (>3 headers) → +1
Rule 6 test/doc/config file → reset score to 0
Rule 7 error-handling patterns → +1 (case-sensitive)
Rule ordering preserved from bash: rule 6 resets to 0, then rule 7
still applies (e.g. a test file with error patterns scores 1).
parseExcludePrefixes(str): string[]
Shared helper — trims, strips CR, drops blank lines.
extractFilePath improvement over bash awk:
Uses 'diff --git a/'.length + indexOf(' b/') instead of greedy
/.*b\//. Fixes silent mis-extraction for paths containing 'b/'
in a directory component (e.g. .github/workflows/ci.yml where
'github/' contains 'b/').
index.ts CLI entry:
node dist/score-risk.js <diffPath> <excludePaths>
Reads diffPath, calls scoreFiles(), writes /tmp/file_risk_scores.json.
Tests: src/score-risk/__tests__/score-risk.test.ts (57 tests)
Every scoring rule tested in isolation with boundary conditions,
combined-rule scenarios (max score = 6), and edge cases.
207/207 total unit tests pass.
tsup.config.ts: added 'score-risk': src('score-risk') to entry map.
Bundles to dist/score-risk.js (3.42 KB, zero npm deps).
review-pr/action.yml: replace 88-line bash block with:
node "$ACTION_PATH/dist/score-risk.js" pr.diff "$EXCLUDE_PATHS"
echo "✅ File risk scores: $(jq -c . /tmp/file_risk_scores.json)"
Step name, if: condition, and env: block are unchanged.
Adds the score-risk/ entry alphabetically between filter-diff/ and get-pr-meta/, consistent with the filter-diff entry added in the previous commit.
… test
[MEDIUM] Add set -euo pipefail to Score file risk step.
Without it, if score-risk.js exits non-zero bash continues to the echo
which exits 0, masking the failure and leaving /tmp/file_risk_scores.json
missing or stale. The pr-review agent reads this file directly, so a
silent failure here could misroute the entire review.
[LOW] Add behavioral regression test for extractFilePath greedy-regex fix.
The existing .github/workflows/ci.yml test produces score 0 under both
old and new extraction (both match \.ya?ml$), so it does not demonstrate
that the fix changes any outcome. The new test uses src/auth/b/helper.ts:
Old replace(/.*b\//): extracts 'helper.ts' → no keyword → score 0
New indexOf(' b/'): extracts 'src/auth/b/helper.ts' → 'auth' → score 2
This proves the new implementation gives a different (correct) result for
paths containing a 'b/' directory component before a security keyword.
4bc6182 to
690ad4e
Compare
Summary
Fixes a timeout issue in the PR review pipeline caused by generated protobuf files dominating the review queue.
Root cause: docker/agentic-platform PR #1965 (4338-line diff) timed out because
session_service.pb.gowas assigned risk score 6 (matchedsessionin the security-path pattern + >100 added lines), landing in Chunk 1 with highest priority. The agent spent ~77k lines processing it with zero findings, leaving no capacity for real code.Fix A —
exclude-pathsinputAdds optional
exclude-pathsinput (newline-separated path prefixes) to both the reusable workflow and the composite action.New step: 'Filter excluded paths from diff' runs after
gh pr diffpre-fetch and before 'Split diff into chunks':+++ b/<path>line is seen⏭️ Excluded from review: <path>✅ Filtered diff: N files excluded, M files remainingpr.diffexists ANDexclude-pathsis non-emptyFix B — Risk scorer ignores excluded paths + generated-file markers
Updated 'Score file risk' bash step:
exclude-pathsprefix → score 0 (redundant safety net if the diff still has those sections for some reason)exclude-paths, inspects first 5 added lines forCode generatedorDO NOT EDIT→ score 0 (catches .pb.go and similar files not covered by a path prefix)Files changed
.github/workflows/review-pr.yml: addedexclude-pathstoworkflow_callinputs and threaded through to thereview-praction callreview-pr/action.yml: addedexclude-pathsinput, new filter step, updated scoring step withEXCLUDE_PATHSenv var and two new zero-score conditionsConsumer configuration
Downstream repos (e.g. docker/agentic-platform) can now configure:
Refs: analysis artifact
86ed46d9-3606-4072-b570-62204d354c86