[repository-quality] 🎯 Repository Quality Improvement Report - Error Message Actionability & Consistency

[github-actions[bot]]

Analysis Date: 2026-08-05
Focus Area: Error Message Actionability & Consistency (custom)
Strategy Type: Custom
Custom Area: Yes — the repository already maintains a dedicated .github/skills/error-messages/SKILL.md style guide and a purpose-built pkg/linters/errormessage analyzer, but both are scoped only to changed files in CI. This leaves a large body of pre-existing errors unaudited. This area is highly repo-specific (tailored conventions + custom linter) rather than a generic "Code Quality" pass, so it was selected as a custom focus area.

Executive Summary

gh-aw enforces an actionable error-message style guide (what's wrong → what's expected → how to fix) and even ships a custom Go analyzer (pkg/linters/errormessage) that flags negative-only wording, unguarded failed to X: %w wrappers, and missing NewValidationError suggestions. However, the linter only runs against files changed in a given CI diff (--changed-files flag), so the vast majority of the ~2,600 error-producing call sites in pkg/ were never checked against the guide. A repo-wide grep sample shows ~1,129 instances of the generic fmt.Errorf("failed to ...: %w", err) pattern the guide explicitly says to avoid "unless you add recovery guidance," plus 51 error strings that violate the Go convention of lowercase, non-punctuated error text.

The greatest opportunity is not writing more prose guidance — the guide is already good — but (1) extending linter coverage from diff-only to a full-repo audit mode usable in scheduled CI or make lint, and (2) fixing the highest-traffic offending files (pkg/cli/pr_command.go, pkg/cli/trial_repository.go, pkg/workflow/compiler_custom_jobs.go) as exemplars other contributors can pattern-match against. Tackling these in small, file-scoped PRs keeps risk low while meaningfully raising the actionability bar for the CLI's user-facing errors.

Full Analysis Report

Focus Area: Error Message Actionability & Consistency

Current State Assessment

The project has strong tooling already: .github/skills/error-messages/SKILL.md defines a three-part template (what's wrong / what's expected / how to fix), a preference for NewValidationError(field, value, reason, suggestion) in *_validation.go files, and an explicit rule against bare fmt.Errorf("failed to X: %w", err) wrapping without recovery guidance. pkg/linters/errormessage/errormessage.go implements checkNegativeLanguage, checkFailedToErrorfWrap, and checkNewValidationSuggestion checks, but is gated by a --changed-files flag and is a no-op with no changed files — meaning it never audits the historical codebase, only new diffs.

Metrics Collected:

Metric	Value	Status
Total fmt.Errorf(...) calls in pkg/	2,518	i️
fmt.Errorf calls without %w wrapping	929	⚠️
Generic unguarded "failed to X: %w" pattern	1,129	⚠️
errors.New(...) calls	270 files w/ at least one	i️
Error strings starting uppercase (Go convention violation)	26 (fmt.Errorf) + 25 (errors.New) = 51	⚠️
NewValidationError(...) call sites (validation-specific errors)	24	✅ (concentrated well in *_validation.go)
Existing custom analyzer for this area	pkg/linters/errormessage (diff-scoped only)	⚠️
panic(...) calls in non-test code	47	⚠️

Findings

Strengths

• A clear, well-documented error-message style guide exists (error-messages skill) with good/bad examples and a suggestion-text checklist.
• A dedicated static analyzer (pkg/linters/errormessage) already codifies these rules as compile-time checks, integrated with nolint directive support and generated-file skipping.
• Validation-specific errors (NewValidationError) are reasonably concentrated in *_validation.go files (e.g., sandbox_validation.go has 25 uses, network_firewall_validation.go has 15), showing the pattern is being followed where it matters most (user-facing YAML validation).

Areas for Improvement

• ⚠️ High: The errormessage linter is diff-scoped (--changed-files flag) and silently no-ops without it — there is no full-repo/CI baseline audit mode, so ~1,100+ pre-existing unguarded "failed to X: %w" errors are invisible to tooling.
• ⚠️ Medium: 51 error strings across fmt.Errorf/errors.New start with a capital letter (e.g., pkg/workflow/awf_config.go:124, pkg/cli/mcp_registry.go:99-107, pkg/cli/pr_helpers.go:20), violating both Go's error-string convention (lowercase, no trailing punctuation) and readability when wrapped by callers.
• ⚠️ Medium: High-traffic CLI/workflow files concentrate many bare wrapper errors with zero recovery guidance, e.g. pkg/cli/pr_command.go (41 Errorf calls), pkg/cli/trial_repository.go (34 calls, several like "failed to clone host repository %s: %w (output: %s)" with no next-step hint), and pkg/workflow/compiler_custom_jobs.go (31 calls, e.g. "failed to convert runs-on to YAML for job '%s': %w").
• ⚠️ Low: 47 panic(...) calls remain in non-test code; some (e.g. pkg/workflow/model_aliases.go:60) already include a good actionable message ("BUG: ... (try 'make build' to rebuild...)") — a pattern worth extending to the rest.

Detailed Analysis

The errormessage linter's checkFailedToErrorfWrap function already detects the exact anti-pattern found 1,129 times in the codebase, but its --changed-files gate means it currently only prevents new regressions rather than remediating existing debt. Making a full-repo audit mode available (e.g., --changed-files=all or a separate go vet/golangci-lint target with --full-repo) would let maintainers track remediation progress as a metric over time, and could be wired into make lint as a non-blocking report initially.

Convention violations (uppercase-starting error strings) are concentrated in pkg/cli/mcp_registry.go (5 occurrences in a 10-line span, lines 99-107) and are easy, mechanical, low-risk fixes — good candidates for a quick top-down pass with the go-codemod skill if a repeatable pattern emerges (e.g., "any string literal passed to errors.New/fmt.Errorf starting uppercase → lowercase first rune").

---

🤖 Tasks for Copilot Agent

NOTE TO PLANNER AGENT: Split the following tasks into individual work items.

Improvement Tasks

Task 1: Add full-repo audit mode to the errormessage linter

Priority: High
Estimated Effort: Medium
Focus Area: Error Message Actionability & Consistency

Description: The pkg/linters/errormessage analyzer only runs when a --changed-files list is provided (see run() in pkg/linters/errormessage/errormessage.go), and is a silent no-op otherwise. Add a mode (e.g., a --changed-files=all sentinel or a new --full-repo flag) that scans every non-generated, non-test Go file in pkg/, so the existing checks (checkNegativeLanguage, checkFailedToErrorfWrap, checkNewValidationSuggestion) can be run as a repo-wide report (non-blocking initially) to establish a baseline and track remediation over time.

Acceptance Criteria:

• New flag/mode added to pkg/linters/errormessage/errormessage.go that bypasses the "no changed files → no-op" early return and checks all applicable files
• shouldCheckFile updated or bypassed appropriately for full-repo mode without breaking the existing diff-scoped behavior
• A make target or CI step documented (non-blocking) to run the audit and print a summary count of violations
• Unit test added in pkg/linters/errormessage/errormessage_test.go covering the new full-repo mode
• go-linters skill guidance followed for analyzer conventions

Code Region: pkg/linters/errormessage/errormessage.go (functions run, parseChangedFiles, shouldCheckFile)

In pkg/linters/errormessage/errormessage.go, the `run` function currently no-ops when `changedFilesCSV` is empty. Add a full-repo audit mode: when the flag is set to a sentinel value like "all" (or add a new boolean flag `--full-repo`), skip the changed-files early return and instead call `shouldCheckFile` such that it returns true for every non-test, non-generated .go file under pkg/ (reusing the existing `filecheck.ShouldSkipFilename` and `nolint` directive support). Keep the default diff-scoped behavior unchanged when no flag is passed. Add a corresponding unit test in errormessage_test.go that exercises the full-repo path, and document how to invoke it (e.g., via a new `make` target) in the pkg/linters/errormessage package or the go-linters skill. Do not make this mode fail CI by default — treat it as a reporting-only audit tool at least initially.

---

Task 2: Fix uppercase-starting error strings in pkg/cli/mcp_registry.go and related files

Priority: Medium
Estimated Effort: Small
Focus Area: Error Message Actionability & Consistency

Description: Multiple error strings violate Go's convention that error strings should not be capitalized (they get wrapped/concatenated by callers). pkg/cli/mcp_registry.go lines 99-107 have five consecutive violations ("MCP registry access forbidden (403): %s...", "MCP registry access unauthorized (401): %s...", etc.). Similar issues exist in pkg/workflow/awf_config.go:124, pkg/workflow/schema_validation.go:108, pkg/cli/forecast_resolution.go:149, pkg/cli/mcp_validation.go:85,99,111, pkg/cli/pr_helpers.go:20, pkg/cli/run_workflow_execution.go:88, pkg/cli/enable.go:73, and others found via grep -rnoP 'fmt\.Errorf\("\K[A-Z]' --include=*.go pkg/ and grep -rnP 'errors\.New\("\K[A-Z]' --include=*.go pkg/.

Acceptance Criteria:

• Lowercase the first letter of each identified error string (keep proper nouns like "GitHub", "MCP", "HTTP", "API" capitalized per Go convention — only the leading word if it's not a proper noun needs lowercasing)
• Verify no message ends in a period (already 0 found, keep it that way)
• Run make fmt and existing errormessage/golangci-lint checks to confirm no regressions
• Add/update nolint directives only if a proper noun exception genuinely can't be reworded

Code Region: pkg/cli/mcp_registry.go:99-107, pkg/workflow/awf_config.go:124, pkg/cli/mcp_validation.go:85,99,111, pkg/cli/pr_helpers.go:20

Find and fix Go error-string convention violations (error strings should start with a lowercase letter and have no trailing punctuation, per Go's official style guidance and this repo's error-messages skill). Focus first on pkg/cli/mcp_registry.go lines 99-107, which contain five fmt.Errorf calls starting with "MCP registry ..." — lowercase "MCP registry" only if not treated as a proper noun exception; if "MCP" must stay capitalized as an acronym, that's fine, but ensure the overall string still reads naturally when wrapped (e.g., "mcp registry access forbidden (403): %s..."). Also fix: pkg/workflow/awf_config.go:124 ("AWF config schema validation failed: %w"), pkg/workflow/schema_validation.go:108 ("GitHub Actions schema validation failed: %w"), pkg/cli/mcp_validation.go:85,99,111 ("GitHub token required..."), and pkg/cli/pr_helpers.go:20 ("GitHub CLI (gh) is required for PR creation but not available"). Use `grep -rnoP 'fmt\.Errorf\("\K[A-Z][^"]*' --include=*.go pkg/` and `grep -rnP 'errors\.New\("\K[A-Z]' --include=*.go pkg/` to find the complete list (51 total across fmt.Errorf and errors.New), excluding _test.go files, and fix all of them consistently. Run `go build ./...` and existing unit tests after changes to confirm no test assertions depended on the old casing.

---

Task 3: Add recovery guidance to high-traffic bare "failed to X: %w" errors in pkg/cli/trial_repository.go

Priority: Medium
Estimated Effort: Medium
Focus Area: Error Message Actionability & Consistency

Description: pkg/cli/trial_repository.go has 34 fmt.Errorf calls, several of which are bare "failed to X: %w" wrappers around git/gh operations with no recovery guidance, violating the error-messages skill's rule to avoid generic wrappers "unless you add recovery guidance." Examples: line 108 "failed to force delete existing host repository %s: %w (output: %s)", line 165 "failed to create host repository: %w (output: %s)", line 214 "failed to delete host repository: %w (output: %s)", line 243 "failed to clone host repository %s: %w (output: %s)".

Acceptance Criteria:

• Each of the identified errors gets an actionable suffix (e.g., permission hints, auth hints like "run 'gh auth login'", or retry guidance) following the style guide's what's-wrong/what's-expected/how-to-fix template
• No behavior change to control flow, only message text
• pkg/linters/errormessage (run in diff mode against this file) passes without new violations
• Existing tests in pkg/cli/trial_repository_test.go (if present) still pass; add/update test assertions on error text where tests check exact strings

Code Region: pkg/cli/trial_repository.go:108,165,214,243,257 (and other fmt.Errorf("failed to ... calls in this file)

In pkg/cli/trial_repository.go, several fmt.Errorf calls wrap git/gh CLI failures with a bare "failed to X: %w" message and no recovery guidance, which the repository's .github/skills/error-messages/SKILL.md explicitly discourages. Update these calls to add actionable context, for example:
- Line 108 ("failed to force delete existing host repository %s: %w (output: %s)") — add a hint about checking repo permissions or manually deleting via `gh repo delete`.
- Line 165 ("failed to create host repository: %w (output: %s)") — add a hint to verify GitHub authentication (`gh auth status`) or organization permissions.
- Line 214 ("failed to delete host repository: %w (output: %s)") — similar permission/auth hint.
- Line 243 ("failed to clone host repository %s: %w (output: %s)") — hint to check network connectivity or repo URL validity.
- Line 257 ("failed to get current directory: %w") — likely an environment issue; note that this is usually transient/unexpected.
Follow the [what's wrong]. [what's expected]. [example of correct usage] template from the error-messages skill. Keep error strings lowercase and unpunctuated per Go convention. After editing, run `go build ./pkg/cli/...` and the package's existing tests to confirm no assertions on exact error text broke; update any that did.

---

Task 4: Add recovery guidance to pkg/workflow/compiler_custom_jobs.go YAML-conversion errors

Priority: Low
Estimated Effort: Small
Focus Area: Error Message Actionability & Consistency

Description: pkg/workflow/compiler_custom_jobs.go has 31 fmt.Errorf calls, several following the pattern "failed to convert X to YAML for job '%s': %w" (e.g., lines 207, 258, 315, 357 for strategy, runs-on, concurrency, and container respectively) with no guidance on what causes a YAML conversion failure or how a workflow author should fix their frontmatter.

Acceptance Criteria:

• Each conversion-failure error includes a hint pointing to the relevant frontmatter field and expected type/shape
• Where feasible, use NewValidationError instead of fmt.Errorf if the error is a genuine user-facing configuration mistake (per the skill's guidance: NewValidationError for *_validation.go-style logic, fmt.Errorf for operational/wrapping errors)
• make recompile run on any .github/workflows/*.md files exercising custom jobs to confirm no regressions
• Existing compiler tests for custom jobs continue to pass

Code Region: pkg/workflow/compiler_custom_jobs.go:53,207,258,315,357

In pkg/workflow/compiler_custom_jobs.go, several fmt.Errorf calls report generic YAML-conversion failures for custom job fields without guidance, e.g.:
- Line 207: "failed to convert strategy to YAML for job '%s': %w"
- Line 258: "failed to convert runs-on to YAML for job '%s': %w"
- Line 315: "failed to convert concurrency to YAML for job '%s': %w"
- Line 357: "failed to convert container to YAML for job '%s': %w"
- Line 53: "failed to add custom job '%s': %w"
These are likely triggered by malformed `strategy`, `runs-on`, `concurrency`, or `container` fields in a workflow's custom `jobs:` frontmatter. Update each message to hint at the expected shape (e.g., "runs-on: expects a string or list of strings, see GitHub Actions docs") and reference the workflow field name so authors can self-diagnose without reading compiler source. Follow the error-messages skill template ([what's wrong]. [what's expected]. [example]). Where the error stems directly from invalid user-authored frontmatter (not an internal bug), consider whether NewValidationError from pkg/workflow/workflow_errors.go would be more appropriate than a bare fmt.Errorf wrap — check how other *_validation.go files in pkg/workflow use it for the field/reason/suggestion parameters. Verify with `make recompile` against any existing custom-jobs test workflow fixtures and run relevant workflow compiler unit tests.

---

📊 Historical Context

Previous Focus Areas

Date	Focus Area	Type	Custom	Key Outcomes
2026-08-04	Code Organization	Standard	No	Identified 15 non-test Go files >1000 lines; proposed splitting largest offenders (awf_helpers.go, update_actions.go, compiler_custom_jobs.go)
2026-08-05	Error Message Actionability & Consistency	Custom	Yes	Found 1,129 unguarded "failed to X: %w" wrappers, 51 Go-convention casing violations, and a diff-scoped-only linter with no full-repo audit mode

---

🎯 Recommendations

Immediate Actions (This Week)

1. Add full-repo audit mode to pkg/linters/errormessage — Priority: High

Short-term Actions (This Month)

1. Fix uppercase-starting error strings (51 occurrences) — Priority: Medium
2. Add recovery guidance to pkg/cli/trial_repository.go top offenders — Priority: Medium

Long-term Actions (This Quarter)

1. Progressively add recovery guidance across all ~1,129 bare "failed to X: %w" wrappers, tracked via the new full-repo audit mode — Priority: Low

---

📈 Success Metrics

• Unguarded "failed to X: %w" wrappers: 1,129 → trending down each quarter (tracked via new audit mode)
• Go convention casing violations: 51 → 0
• errormessage linter coverage: diff-only → full-repo audit mode available

---

Next Steps

1. Review and prioritise the tasks above
2. Assign tasks to Copilot coding agent via planner agent
3. Track progress on improvement items
4. Re-evaluate this focus area in 1-2 quarters (after remediation tasks land)

---

Generated by Repository Quality Improvement Agent
Next analysis: 2026-08-06 — Focus area selected by diversity algorithm

Generated by ⚡ Repository Quality Improvement Agent · auto · 38.3 AIC · ⌖ 4.91 AIC · ⊞ 9.6K · ◷

• expires on Aug 6, 2026, 5:28 AM UTC-08:00