[repository-quality] 🎯 Repository Quality Improvement Report - Error Message Actionability Debt

[github-actions[bot]]

Analysis Date: 2026-08-26
Focus Area: Error Message Actionability Debt (non-actionable/generic fmt.Errorf wrapping)
Strategy Type: Custom
Custom Area: Yes — the repo already ships a dedicated errormessage Go analyzer (pkg/linters/errormessage/) and a style guide (.github/skills/error-messages/SKILL.md), but the linter is only enforced on changed files in CI (.github/workflows/error-message-lint.yml) while a non-blocking, full-repo audit target exists (make lint-error-messages-report) whose findings are never tracked or actioned. This creates a large, growing pool of legacy violations that new code inherits by proximity (copy-paste) even though the gate prevents new violations in touched lines.

Executive Summary

Running make lint-error-messages-report (full-repo mode of the existing errormessage analyzer) surfaces 2,631 violations across 344 Go files, split between two rule classes: 1,582 "negative language without constructive guidance" findings (e.g. bare invalid X, not found, cannot Y with no expected format or example) and 1,049 "generic failed to ...: %w wrapping" findings that swallow context a caller could act on. The debt is heavily concentrated in pkg/cli (1,951 hits, 74%), with pkg/workflow (621) and pkg/parser (227) next. Single files such as pkg/cli/project_command.go (56), pkg/cli/trial_repository.go (55), and pkg/cli/includes.go (45) account for a disproportionate share, making them efficient targets for concentrated cleanup.

Separately, 82 *_validation.go files exist repo-wide but only 29 use the repo's own NewValidationError(field, value, reason, suggestion) helper — meaning 53 validation files (mostly in pkg/workflow) raise raw errors.New/fmt.Errorf for user-facing YAML/config validation instead of the structured, example-driven format the style guide mandates. This matters more than generic internal errors because these are the messages workflow authors see directly when gh aw compile rejects their frontmatter.

The recommended path is not to rewrite thousands of messages by hand, but to (1) promote the full-repo audit from silent/non-blocking to a tracked metric with a ratchet (fail CI if violation count increases), (2) sweep the top 5 highest-violation-count pkg/cli files as a concentrated first pass, and (3) migrate the highest-traffic pkg/workflow/*_validation.go files that still use raw errors to NewValidationError, since these are the most user-visible.

Full Analysis Report

Focus Area: Error Message Actionability Debt

Current State Assessment

The errormessage analyzer (pkg/linters/errormessage/errormessage.go) already implements two checks:

1. Negative-only wording without expected/should/example guidance
2. Generic fmt.Errorf("failed to X: %w", err) wrapping without recovery guidance

It runs in two modes:

• Blocking, changed-files-only in .github/workflows/error-message-lint.yml (-errormessage.changed-files="$CHANGED_GO_FILES") — prevents new PRs from adding more violations in the lines they touch.
• Non-blocking, full-repo audit via make lint-error-messages-report (-errormessage.full-repo, piped through || true) — surfaces legacy debt but is not wired into any CI gate, dashboard, or ratchet, so the count has no visible trend and no owner.

Metrics Collected:

Metric	Value	Status
Total errormessage violations (full-repo audit)	2,631	❌
Files with ≥1 violation	344	⚠️
"Negative language, no guidance" violations	1,582 (60%)	⚠️
"Generic failed to ...: %w" violations	1,049 (40%)	⚠️
Violations in pkg/cli	1,951 (74%)	❌
Violations in pkg/workflow	621 (24%)	⚠️
*_validation.go files total	82	—
*_validation.go files using NewValidationError	29 (35%)	❌
*_validation.go files using raw errors.New/fmt.Errorf	53 (65%)	❌
Full-repo audit wired into blocking CI	No	❌

Findings

Strengths

• A purpose-built static analyzer (pkg/linters/errormessage) and a clear style guide (.github/skills/error-messages/SKILL.md) already exist — no new tooling is required, only better enforcement and adoption.
• The changed-files gate (error-message-lint.yml) is already blocking, so violation growth from new code is capped at the margin.
• A NewValidationError(field, value, reason, suggestion) helper exists in both pkg/parser/validation_error.go and pkg/workflow/workflow_errors.go, giving a consistent target pattern to migrate toward.

Areas for Improvement

• [High] The full-repo audit (2,631 violations) is invisible to CI — no metric is tracked, so the debt cannot regress or improve visibly; make lint-error-messages-report always exits 0 via || true.
• [High] 65% of *_validation.go files (53/82) — mostly in pkg/workflow, e.g. glob_validation.go, secrets_validation.go, permissions_validation.go, docker_validation.go — raise raw errors instead of NewValidationError, directly affecting user-facing YAML validation messages seen by workflow authors.
• [Medium] Violation debt is extremely concentrated: just 15 files (project_command.go, trial_repository.go, includes.go, setup_repository.go, git.go, interactive.go, remote_list_files.go, graders_operational_value_regrade.go, remote_download_file.go, update_workflows.go, trial_helpers.go, compiler_safe_output_jobs.go, init.go, run_push.go, download_workflow.go) account for ~550 of the 2,631 violations (21%), making them the highest-leverage cleanup targets.
• [Low] No trend/ratchet mechanism exists to prevent the full-repo count from silently growing between now and any future adoption push.

Detailed Analysis

The 40% "generic wrapping" class (fmt.Errorf("failed to X: %w", err)) is typically the cheapest to fix: appending a short recovery clause (e.g., "— check permissions" or "— verify the file exists at path") satisfies the linter without restructuring logic. The 60% "negative language" class needs slightly more care since it requires adding an expected-format clause and, ideally, a concrete example per the style guide's [what's wrong]. [what's expected]. [example] template.

Given the 74% concentration in pkg/cli, a natural rollout order is: (1) ratchet the audit into CI to stop growth, (2) sweep the top 15 files for maximum debt reduction per file touched, (3) migrate pkg/workflow/*_validation.go files to NewValidationError since they are the most user-visible (workflow authors see these in gh aw compile output), lower remaining pkg/parser and pkg/cli volume opportunistically in follow-up passes.

---

🤖 Tasks for Copilot Agent

NOTE TO PLANNER AGENT: Split the following tasks into individual work items.

Improvement Tasks

Task 1: Add a CI ratchet for the full-repo error-message audit

Priority: High
Estimated Effort: Small
Focus Area: Error Message Actionability Debt

Description: Wire make lint-error-messages-report into a CI job that fails if the full-repo violation count increases versus a tracked baseline (store the current count, e.g. 2,631, in a small baseline file such as .github/error-message-baseline.txt), so new debt cannot accumulate silently while existing debt is paid down incrementally.

Acceptance Criteria:

• Baseline violation count file added and checked into the repo
• New CI step (or extension of error-message-lint.yml) runs the full-repo audit, counts violations, and fails if count > baseline
• CI step allows lowering the baseline when violations are fixed (documented in a short comment or Makefile target)
• make lint-error-messages-report output format unchanged for local dev use

Code Region: .github/workflows/error-message-lint.yml, Makefile (lint-error-messages-report target)

Add a CI ratchet for the `errormessage` full-repo audit in gh-aw.

Context: `pkg/linters/errormessage` already implements a Go analyzer with `-full-repo` mode that reports non-actionable error message patterns (negative-only wording, generic `failed to ...: %w` wrapping) across the whole repo. `make lint-error-messages-report` runs this in full-repo mode but always exits 0 (`|| true`), so results are never enforced. `.github/workflows/error-message-lint.yml` only gates changed files via `-errormessage.changed-files`.

Task: introduce a baseline file (e.g. `.github/error-message-baseline.txt`) containing the current full-repo violation count (run `make lint-error-messages-report` to get the current total via `wc -l`, currently ~2,631). Add a new Makefile target (e.g. `lint-error-messages-ratchet`) that reruns the full-repo audit, counts violations, compares to the baseline, and exits non-zero if the count increased, printing a clear diff-style message. Wire this new target into `.github/workflows/error-message-lint.yml` as an additional step (or a new job) so PRs that increase the debt count fail CI, while PRs that reduce it succeed and can optionally regenerate the baseline file. Document how to update the baseline in a short comment in the Makefile target itself.

Keep the existing changed-files-only gate untouched — this is an additive ratchet, not a replacement.

---

Task 2: Migrate high-traffic pkg/workflow/*_validation.go files to NewValidationError

Priority: High
Estimated Effort: Medium
Focus Area: Error Message Actionability Debt

Description: 53 of 82 *_validation.go files across the repo raise raw errors.New/fmt.Errorf instead of using the existing NewValidationError(field, value, reason, suggestion) helper (defined in pkg/workflow/workflow_errors.go and pkg/parser/validation_error.go). These are user-facing messages shown when gh aw compile rejects invalid workflow frontmatter, so they benefit most from the structured [what's wrong]. [what's expected]. [example] format. Migrate the highest-traffic offenders first: pkg/workflow/secrets_validation.go, pkg/workflow/glob_validation.go, pkg/workflow/permissions_validation.go, pkg/workflow/docker_validation.go, pkg/workflow/engine_validation.go.

Acceptance Criteria:

• Each targeted file's raw errors.New/fmt.Errorf validation errors converted to NewValidationError(field, value, reason, suggestion) calls with concrete example suggestions
• Existing unit tests for these validation files updated/passing (go test ./pkg/workflow/... -run Validation)
• make lint-error-messages-report violation count for these 5 files reduced to 0 (verify via targeted grep of the audit output)
• No behavioral change to validation logic — only message construction

Code Region: pkg/workflow/secrets_validation.go, pkg/workflow/glob_validation.go, pkg/workflow/permissions_validation.go, pkg/workflow/docker_validation.go, pkg/workflow/engine_validation.go

Migrate raw error construction to `NewValidationError` in 5 high-traffic gh-aw workflow validation files.

Context: gh-aw has a structured validation error helper `NewValidationError(field, value, reason, suggestion string) *WorkflowValidationError` in `pkg/workflow/workflow_errors.go`. The style guide at `.github/skills/error-messages/SKILL.md` mandates using this helper in `*_validation.go` files, with `suggestion` including a concrete valid example. However, `pkg/workflow/secrets_validation.go`, `pkg/workflow/glob_validation.go`, `pkg/workflow/permissions_validation.go`, `pkg/workflow/docker_validation.go`, and `pkg/workflow/engine_validation.go` currently raise raw `errors.New(...)` / `fmt.Errorf(...)` for user-facing YAML validation failures (e.g. `secrets_validation.go` line ~24: `errors.New("invalid secrets expression: must be a GitHub Actions expression with secrets reference (e.g., '${{ secrets.MY_SECRET }}' or '${{ secrets.SECRET1 || secrets.SECRET2 }}')")`).

Task: for each of these 5 files, replace raw error construction with `NewValidationError(field, value, reason, suggestion)`, splitting the existing message into: `field` (the YAML/config key being validated), `value` (the invalid value, if available), `reason` (what's wrong), and `suggestion` (what's expected + a concrete ✓ example, following the template in `.github/skills/error-messages/SKILL.md`). Preserve all existing validation logic and error triggers — only change how the error message is constructed. Run `go test ./pkg/workflow/...` after each file to confirm no test regressions (update test assertions on error message text if they check exact strings). After changes, run `make fmt` and `make recompile` if any `.md` workflow files were touched (they should not be for this task).

---

Task 3: Sweep the top 5 pkg/cli files by error-message violation count

Priority: Medium
Estimated Effort: Medium
Focus Area: Error Message Actionability Debt

Description: pkg/cli/project_command.go (56 violations), pkg/cli/trial_repository.go (55), pkg/cli/includes.go (45), pkg/cli/setup_repository.go (40), and pkg/cli/git.go (40) together account for ~236 of the 2,631 total full-repo violations (9%). Fix these five files' fmt.Errorf/errors.New calls per the style guide: add expected-format context and a recovery suggestion, and avoid bare failed to X: %w wrapping.

Acceptance Criteria:

• make lint-error-messages-report shows 0 violations for all 5 named files
• Error message rewrites follow .github/skills/error-messages/SKILL.md template ("what's wrong. what's expected. example")
• go build ./... and relevant go test ./pkg/cli/... pass unchanged aside from message text
• No functional/logic changes — text-only edits to error message strings

Code Region: pkg/cli/project_command.go, pkg/cli/trial_repository.go, pkg/cli/includes.go, pkg/cli/setup_repository.go, pkg/cli/git.go

Fix non-actionable error messages in the top 5 highest-violation `pkg/cli` files in gh-aw.

Context: gh-aw's `errormessage` Go analyzer (`pkg/linters/errormessage/`) flags two patterns: (1) negative-only wording without expected-format/example guidance, and (2) generic `fmt.Errorf("failed to X: %w", err)` wrapping without recovery context. Running `make lint-error-messages-report` (full-repo audit mode) shows `pkg/cli/project_command.go` has 56 violations, `pkg/cli/trial_repository.go` has 55, `pkg/cli/includes.go` has 45, `pkg/cli/setup_repository.go` has 40, and `pkg/cli/git.go` has 40 — the highest concentration of any files in the repo.

Task: run `make lint-error-messages-report 2>&1 | grep -E "project_command.go|trial_repository.go|includes.go:|setup_repository.go|pkg/cli/git.go"` to get exact line numbers for each violation in these 5 files. For each flagged `fmt.Errorf`/`errors.New` call, rewrite the message following the style guide at `.github/skills/error-messages/SKILL.md`: state what's wrong, what's expected, and (where practical) a short example, replacing bare `failed to X: %w` wrapping with wrapping that adds a specific recovery hint (e.g., "failed to clone repository %q: %w — verify network access and that the repo exists" instead of just "failed to clone: %w"). Do not change control flow or logic — only the error message text. After edits, re-run `make lint-error-messages-report` and confirm these 5 files no longer appear in the output, then run `go build ./...` to verify compilation.

---

📊 Historical Context

Previous Focus Areas

Date	Focus Area	Type	Custom	Key Outcomes
2026-08-24	Large File Decomposition Debt	Custom	Y	32 non-test files over 800 lines, concentrated in pkg/cli (15) and pkg/workflow (12)
2026-08-25	Dependency Health & Vulnerability Posture	Standard	N	116 total deps, 100 outdated, 0 reachable vulns, 1 non-reachable advisory (GO-2026-5932)
2026-08-26	Error Message Actionability Debt	Custom	Y	2,631 full-repo violations across 344 files; audit exists but not CI-enforced beyond changed files

---

🎯 Recommendations

Immediate Actions (This Week)

1. Add the CI ratchet for the full-repo error-message audit — Priority: High
2. Sweep the top 5 pkg/cli violation files — Priority: Medium

Short-term Actions (This Month)

1. Migrate pkg/workflow/*_validation.go files to NewValidationError — Priority: High
2. Extend the sweep to the next 10 highest-violation files (interactive.go, remote_list_files.go, graders_operational_value_regrade.go, remote_download_file.go, update_workflows.go, trial_helpers.go, compiler_safe_output_jobs.go, init.go, run_push.go, download_workflow.go) — Priority: Medium

Long-term Actions (This Quarter)

1. Drive the full-repo violation count toward 0 via the ratchet, one PR at a time — Priority: Low
2. Consider auto-fix tooling (Go codemod, per the go-codemod skill) for the most mechanical "generic wrapping" class — Priority: Low

---

📈 Success Metrics

• Full-repo errormessage violations: 2,631 → 0 (tracked via ratchet baseline)
• *_validation.go files using NewValidationError: 29/82 (35%) → 82/82 (100%)

---

Next Steps

1. Review and prioritise the tasks above
2. Assign tasks to Copilot coding agent via planner agent
3. Track progress on improvement items
4. Re-evaluate this focus area in 1 month once the ratchet has driven initial reductions

---

Generated by Repository Quality Improvement Agent
Next analysis: 2026-08-27 — Focus area selected by diversity algorithm

Generated by ⚡ Repository Quality Improvement Agent · copilot · auto · 40.2 AIC · ⌖ 5.89 AIC · ⊞ 9.9K · ◷

• expires on Aug 27, 2026, 5:18 AM UTC-08:00