fix: distinguish actionlint integration failures from lint findings#20637
fix: distinguish actionlint integration failures from lint findings#20637
Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds explicit tracking and reporting of actionlint tooling/integration failures so users aren’t misled by “No issues found” when actionlint couldn’t run/parse successfully.
Changes:
- Extend
ActionlintStatswithIntegrationErrorsand update the summary output to warn on integration failures (with/without lint findings). - Track integration/tooling failures in
runActionlintOnFile(timeouts, non-parseable output, non-1 exit codes, invocation errors). - Add regression tests ensuring integration failures never print “No issues found”, and update stats initialization test.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| pkg/cli/actionlint.go | Adds IntegrationErrors tracking and updates summary + actionlint execution error handling to distinguish tooling failures from lint findings. |
| pkg/cli/actionlint_test.go | Adds regression coverage for summary behavior with integration failures; asserts IntegrationErrors initializes to 0. |
Comments suppressed due to low confidence (1)
pkg/cli/actionlint.go:288
- IntegrationErrors can be double-counted for a single invocation. For example, if stdout is unparseable (parseErr != nil) and the process also exits with a non-1 exit code later in this function, IntegrationErrors is incremented here and again in the non-1 exit-code/other error branches. Consider tracking a per-invocation boolean (or consolidating the increment) so each actionlint run contributes at most 1 IntegrationErrors increment.
totalErrors, errorsByKind, parseErr := parseAndDisplayActionlintOutput(stdout.String(), verbose)
if parseErr != nil {
actionlintLog.Printf("Failed to parse actionlint output: %v", parseErr)
// Track this as an integration error: output was produced but could not be parsed
if actionlintStats != nil {
actionlintStats.IntegrationErrors++
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
| if len(lockFiles) == 1 { | ||
| fileList = filepath.Base(lockFiles[0]) | ||
| } | ||
| if actionlintStats != nil { |
There was a problem hiding this comment.
In the timeout path, IntegrationErrors is incremented but TotalWorkflows is not. Since displayActionlintSummary currently returns early when TotalWorkflows == 0, a run where all actionlint invocations time out (or fail before TotalWorkflows is incremented) will produce no summary at all, even though integration failures occurred. Consider incrementing TotalWorkflows for attempted files before running (or in this timeout branch), and/or adjusting the summary guard to also render when IntegrationErrors > 0.
This issue also appears on line 282 of the same file.
| if actionlintStats != nil { | |
| if actionlintStats != nil { | |
| // Count workflows we attempted to validate before timing out | |
| actionlintStats.TotalWorkflows += len(lockFiles) |
Add glossary entries for the three security scanner tools surfaced by compile flags (--actionlint, --zizmor, --poutine). These tools are referenced in compilation docs but had no glossary definitions. Triggered by PR #20637 which improved actionlint integration error reporting. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
compile --actionlintcould show "No issues found" while exiting nonzero, because Docker/subprocess failures, JSON parse errors, and timeouts leftTotalErrorsat 0 — indistinguishable from a clean run. In strict mode, a parse failure produced the confusing message "found 0 errors" despite a nonzero exit.Changes
ActionlintStats— newIntegrationErrorscountercommand not found, and timeouts all increment this counterrunActionlintOnFile— explicit integration failure messagesFormatWarningMessagefor each failure class explaining it's a tooling error, not a workflow problemparseErr != nil+ exit code 1: was"found 0 errors", now"output could not be parsed — this is likely a tooling or integration error"displayActionlintSummary— accurate messaging"No issues found"is only shown when bothTotalErrors == 0andIntegrationErrors == 0IntegrationErrors > 0with no lint findings:IntegrationErrors > 0alongside lint findings, both are reportedTests
"No issues found"; integration errors alongside findings must report bothTestInitActionlintStatsverifiesIntegrationErrorsstarts at 0Warning
Firewall rules blocked me from connecting to one or more addresses (expand for details)
I tried to connect to the following addresses, but was blocked by firewall rules:
https://api.github.com/graphql/usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw GO111MODULE 64/bin/go git rev-�� --show-toplevel go /usr/bin/git -json GO111MODULE 64/bin/go git(http block)https://api.github.com/repos/actions/ai-inference/git/ref/tags/v1/usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq .object.sha --show-toplevel ache/go/1.25.0/xGO111MODULE e/git 202269/b409/_pkggit GO111MODULE 64/bin/go e/git rev-�� om/owner/repo.git om/owner/repo.git /opt/hostedtoolcache/node/24.14.0/x64/bin/node vONO/owNZP0Uu6SQgit GO111MODULE 64/bin/go node(http block)https://api.github.com/repos/actions/checkout/git/ref/tags/v3/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq .object.sha FVHF/Ptsx3zT9f9y_24ToFVHF GO111MODULE ache/node/24.14.0/x64/bin/node GOINSECURE GOMOD GOMODCACHE 3403864/b372/importcfg t-91�� sistency_GoAndJavaScript202051374/001/test-frontmatter-with-nest-s GO111MODULE 0/x64/bin/node GOINSECURE GOMOD GOMODCACHE 0/x64/bin/node(http block)https://api.github.com/repos/actions/checkout/git/ref/tags/v5/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha --show-toplevel 202269/b394/impo-tests /usr/bin/git k/gh-aw/gh-aw/pkgit k/gh-aw/gh-aw/pkrev-parse 64/bin/go git rev-�� --git-dir /opt/hostedtoolc-buildtags /usr/bin/git /tmp/go-build598git -trimpath 64/bin/go /usr/bin/git(http block)/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha --show-toplevel go /usr/bin/git 4/001/test-frontgit GO111MODULE x_amd64/vet git rev-�� --show-toplevel x_amd64/vet /usr/bin/git -json GO111MODULE ache/go/1.25.0/x--show-toplevel git(http block)https://api.github.com/repos/actions/checkout/git/ref/tags/v6/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha --show-toplevel 3403864/b422/_testmain.go /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/link ck '**/*.cjs' '*git GO111MODULE 64/bin/go /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/link -o /tmp/go-build1893403864/b422/styles.test l(http block)/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha --show-toplevel go /opt/hostedtoolcache/node/24.14.0/x64/bin/node ck '**/*.cjs' '*git GO111MODULE 64/bin/go node /tmp�� /tmp/TestHashConsistency_GoAndJavaScript202051374/001/test-complex-frontmatter-with-tools.md ache/go/1.25.0/xGO111MODULE /usr/bin/git 202269/b384/_pkggit GO111MODULE 64/bin/go git(http block)/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha --show-toplevel go /usr/bin/git -json GO111MODULE 64/bin/go git rev-�� --show-toplevel go /usr/bin/git -json GO111MODULE .test git(http block)https://api.github.com/repos/actions/github-script/git/ref/tags/v8/usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha GOSUMDB GOWORK 64/bin/go GOINSECURE GOMOD GOMODCACHE go ache�� -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)/usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha -c=4 -nolocalimports -importcfg /tmp/go-build1893403864/b385/importcfg -embedcfg /tmp/go-build1893403864/b385/embedcfg -pack env 202269/b401/_pkgGOINSECURE GO111MODULE 64/bin/go GOINSECURE b/gh-aw/pkg/pars-atomic GOMODCACHE go(http block)/usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha a1e46e74a3655a0aGOINSECURE GO111MODULE 64/bin/go GOINSECURE erignore GOMODCACHE ache/go/1.25.0/xGO111MODULE env 202269/b412/_pkgGOINSECURE GO111MODULE 64/bin/go GOINSECURE b/gh-aw/pkg/test-o GOMODCACHE go(http block)https://api.github.com/repos/actions/setup-go/git/ref/tags/v4/usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq .object.sha /tmp/gh-aw-test-runs/20260312-060035-27585/test-3823217530/.github/workflows rev-parse /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/link ck '**/*.cjs' '*git GO111MODULE 64/bin/go /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/link -o /tmp/go-build1893403864/b419/stringutil.test -importcfg /usr/bin/git -s -w -buildmode=exe git(http block)https://api.github.com/repos/actions/setup-node/git/ref/tags/v4/usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq .object.sha /tmp/gh-aw-test-runs/20260312-060035-27585/test-713162983/.github/workflows config /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/compile remote.origin.urgit GO111MODULE 64/bin/go /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/compile -o /tmp/go-build1893403864/b419/_pkg_.a -trimpath ache/node/24.14.0/x64/bin/node -p main -lang=go1.25 ache/node/24.14.0/x64/bin/node(http block)https://api.github.com/repos/actions/upload-artifact/git/ref/tags/v4/usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v4 --jq .object.sha -json GO111MODULE 3403864/b405/mathutil.test GOINSECURE GOMOD GOMODCACHE 3403864/b405/mathutil.test e=/t�� 0035-27585/test-1638029594 GO111MODULE(http block)https://api.github.com/repos/github/gh-aw/actions/runs/1/artifacts/usr/bin/gh gh run download 1 --dir test-logs/run-1 GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/actions/runs/12345/artifacts/usr/bin/gh gh run download 12345 --dir test-logs/run-12345 GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/actions/runs/12346/artifacts/usr/bin/gh gh run download 12346 --dir test-logs/run-12346 GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/actions/runs/2/artifacts/usr/bin/gh gh run download 2 --dir test-logs/run-2 GO111MODULE x_amd64/link GOINSECURE GOMOD GOMODCACHE x_amd64/link env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE wJ/GFtmVa307QDDNuUCuh0B/-q2laYTO--jq(http block)https://api.github.com/repos/github/gh-aw/actions/runs/3/artifacts/usr/bin/gh gh run download 3 --dir test-logs/run-3 GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/actions/runs/4/artifacts/usr/bin/gh gh run download 4 --dir test-logs/run-4 GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/actions/runs/5/artifacts/usr/bin/gh gh run download 5 --dir test-logs/run-5 GO111MODULE x_amd64/link GOINSECURE GOMOD GOMODCACHE x_amd64/link env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE VM/p1cMD6_YVq3t5qtiVmRd/sS5Qz_Nzwg9kiACG8oQk(http block)https://api.github.com/repos/github/gh-aw/actions/workflows/usr/bin/gh gh workflow list --json name,state,path GOSUMDB GOWORK 64/bin/go GOINSECURE GOMOD GOMODCACHE go env ck 'scripts/**/*GOINSECURE GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.0.0/usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.2.3/usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE node(http block)https://api.github.com/repos/github/gh-aw/git/ref/tags/v2.0.0/usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq .object.sha e=false GO111MODULE 64/bin/go GOINSECURE %H %ct %D GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE sh(http block)/usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE sh(http block)/usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq .object.sha 202269/b394/_pkgGOINSECURE GO111MODULE 64/bin/go GOINSECURE b/gh-aw/pkg/file-atomic GOMODCACHE go env dyDE/oOxcY6fbL1B-errorsas GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE 202269/b394/impo-tests(http block)https://api.github.com/repos/github/gh-aw/git/ref/tags/v3.0.0/usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE sh(http block)https://api.github.com/repos/nonexistent/action/git/ref/tags/v999.999.999/usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env 4222518365/.github/workflows GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/nonexistent/repo/actions/runs/12345/usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion GOINSECURE GOMOD GOMODCACHE go estl�� -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/owner/repo/actions/workflows/usr/bin/gh gh workflow list --json name,state,path --repo owner/repo 64/bin/go GOINSECURE GOMOD GOMODCACHE go m/_n�� -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)/usr/bin/gh gh workflow list --json name,state,path --repo owner/repo 64/bin/go GOINSECURE GOMOD ode-gyp-bin/sh go m/_n�� -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/owner/repo/contents/file.md/tmp/go-build1893403864/b383/cli.test /tmp/go-build1893403864/b383/cli.test -test.testlogfile=/tmp/go-build1893403864/b383/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true GOINSECURE GOMOD ode-gyp-bin/sh rtcfg k/gh�� ternal/tools/actGOINSECURE GO111MODULE 64/bin/go GOINSECURE GOMOD erignore ache/go/1.25.0/xGO111MODULE(http block)https://api.github.com/repos/test-owner/test-repo/actions/secrets/usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name GOSUMDB GOWORK 64/bin/go GOINSECURE GOMOD GOMODCACHE go env ck 'scripts/**/*GOINSECURE GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)If you need me to access, download, or install something from one of these locations, you can either:
✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.