Add action build verification script

[potiuk]

Summary

• Add utils/verify-action-build.py — a Python script that verifies compiled JavaScript in GitHub Actions matches a clean rebuild from source
• The script runs the build in an isolated Docker container, beautifies both versions of the JS, and shows colored terminal diffs for any differences
• Add --check-dependabot-prs mode for batch-reviewing and merging open dependabot PRs
• Add reviewer instructions to README.md explaining how to use the tool
• Add a checklist item to the PR template prompting reviewers to verify compiled JS

Single action verification

uv run utils/verify-action-build.py org/repo@commit_hash

1. Clones the action at the specified commit in a node:20-slim Docker container
2. Saves the original dist/ files, then deletes compiled JS
3. Rebuilds from source (supports npm, yarn, and pnpm via corepack)
4. Beautifies both versions with jsbeautifier for readable comparison
5. Shows colored diff output with clickable terminal links to GitHub
6. Automatically skips vendored @vercel/ncc runtime helpers (e.g. sourcemap-register.js) when they are not referenced by other JS files
7. Detects non-minified JS (e.g. ncc build without --minify) and explains that rebuild differences are expected due to toolchain versions
8. Detects non-JS actions (docker, composite) and skips build verification with a clear message
9. If previously approved versions exist, shows approval info (who, when, via which PR) and offers to diff source changes
10. Source diffs exclude generated content (dist/, node_modules/, lock files, __tests__/, __mocks__/, .github/) with user confirmation

Batch dependabot PR review (--check-dependabot-prs)

uv run utils/verify-action-build.py --check-dependabot-prs

1. Lists all open dependabot PRs
2. Excludes PRs with "changes requested" reviews or failed CI checks (explains why)
3. For each eligible PR, extracts the action org/repo@hash from the diff
4. Runs the full build verification
5. If verification passes, asks whether to approve and merge
6. On merge, adds a review comment documenting: who reviewed, what was verified, that all checks passed and no changes were requested

Dependencies

• docker (for isolated builds)
• uv (for running the script with inline dependencies)
• gh (GitHub CLI, for --check-dependabot-prs mode)
• Python >= 3.11

Test plan

• Verified with dorny/test-reporter — index.js matches rebuild
• Verified sourcemap-register.js correctly skipped as unreferenced vendored helper
• Verified docker/build-push-action (yarn-based) rebuilds correctly
• Verified dorny/paths-filter (non-minified JS) handled gracefully
• Verified crowdin/github-action (docker action) detected and skipped
• Verified dawidd6/action-send-mail (no dist/, ships node_modules) falls back to source JS
• Verified biomejs/setup-biome (action.yaml instead of action.yml) detected correctly
• Verified opentofu/setup-opentofu source diff shows actual source changes, not compiled dist
• Tested --check-dependabot-prs listing and filtering
• Test full dependabot merge flow end-to-end

Generated-by: Claude Opus 4.6 (1M context)

[potiuk]

Example output of the tool:

When you choose to show diff:

And actual diff:

[potiuk]

OK. The verification script is now battle-tested on some 15 PRS of dependabot. It allows to literally review 100s of actions per hour with the current optimizations.

[potiuk]

Looking for reviews and merge :)

[potiuk]

BTW. @raboof @dfoulks1 -> you can use/test it with:

gh pr co 561
cd utils 
uv run verify-action-build.py --check-dependabot-prs

This should allow you to verify the new dependabot prs and merge them if verification passes.

[potiuk]

We have 3 of them open since yesterday - one of them is not really mergable because it failed actions check - because it uses gradle/actions/wrapper-validation@v3.5.0 without hash-commit pinning. But the other two should be easily verifiable with the tool.

BTW. I will report the gradle/actions/wrapper-validation@v3.5.0 to the action itself - they should fix it.

[dave2wave]

It would be helpful to include a workflow that runs the utility. Currently you have to install and run locally.

[dave2wave]

Looks good to me, but some of the negative results need to be investigated.

[raboof]

This unlocks checking that compiled javascript in 'dist' corresponds to the more auditable source code, which is a significant improvement.

I tested with utils/verify-action-build.py coursier/setup-action@fd1707a76b027efdfb66ca79318b4d29b72e5a02 (for #543 even though that was already merged) and that verified dist/index.js . I have not done a negative test.

I have some more thoughts but will put those into issues :)