[repository-quality] 🎯 Repository Quality Improvement Report - Error Message & Diagnostics Quality

[github-actions[bot]]

Analysis Date: 2026-08-06
Focus Area: Error Message & Diagnostics Quality
Strategy Type: Custom
Custom Area: Yes — gh-aw already codifies an error-message style guide (.github/skills/error-messages/SKILL.md) requiring "what's wrong / what's expected / example" and constructive phrasing, but a repo-wide audit shows widespread drift from that guide across pkg/cli, pkg/workflow, and pkg/parser. Since gh-aw error output is the primary UX surface for CLI users debugging failing workflow compiles, closing this gap has outsized value.

Executive Summary

gh-aw defines 2,403 fmt.Errorf call sites across pkg/parser, pkg/workflow, and pkg/cli, plus 106 uses of the structured NewValidationError helper. The repository's own style guide asks contributors to avoid generic wrappers like fmt.Errorf("failed to X: %w", err) unless recovery guidance is added, and to make invalid ... messages constructive (state what's expected and give an example). A scan found 257 call sites matching the discouraged "failed to <action>: %w" generic-wrapper pattern (e.g., "failed to marshal JSON: %w" appears 13 times, "failed to get current directory: %w" 8 times, "failed to find git root: %w" 8 times), and 215 bare invalid ... messages that do not mention "expected", "for example", or a ✓ marker as the guide requires.

These are heavily concentrated in pkg/cli (176 files touch the generic-wrapper pattern) versus pkg/workflow (66) and pkg/parser (14) — meaning the CLI's user-facing surface is where drift is worst, directly impacting first-run and debugging UX for gh aw compile/gh aw run. Several of the worst offenders are duplicated verbatim across many files (e.g., "failed to get current directory" appears 8 times, "failed to find git root" 8 times), suggesting a shared helper could both de-duplicate the logic and improve messages once instead of per call-site.

Recommended actions: (1) introduce small reusable error helpers in pkg/gitutil and pkg/cli for the most duplicated low-value wraps (current directory, git root, home directory, temp directory) that add one-line recovery guidance; (2) run a targeted sweep of the invalid %s: %w / bare invalid messages in pkg/workflow validation code to convert them to NewValidationError with a suggestion field per the guide; (3) add a lightweight CI/lint check (or extend an existing custom linter) that flags new fmt.Errorf("failed to ...: %w") additions without accompanying context, nudging future PRs toward compliance.

Full Analysis Report

Focus Area: Error Message & Diagnostics Quality

Current State Assessment

The codebase has an explicit style guide (.github/skills/error-messages/SKILL.md) mandating: [what's wrong]. [what's expected]. [example], constructive (non-bare-negative) phrasing, and use of NewValidationError(field, value, reason, suggestion) for config validation versus fmt.Errorf for operational/wrapping errors. This audit measured actual compliance against that guide using static grep-based heuristics (may include some false positives, e.g., legitimately unavoidable low-level I/O wraps).

Metrics Collected:

Metric	Value	Status
Total fmt.Errorf call sites (pkg/parser, pkg/workflow, pkg/cli)	2,403	i️
NewValidationError structured-error usages	106	⚠️ (low relative to Errorf volume)
Generic "failed to X: %w" wrapper call sites (discouraged per guide)	257	❌
Bare invalid ... messages missing "expected"/"for example"/✓	215	❌
Files in pkg/cli with generic-wrapper pattern	176	❌
Files in pkg/workflow with generic-wrapper pattern	66	⚠️
Files in pkg/parser with generic-wrapper pattern	14	✅ (relatively low)
Most-duplicated generic wrap: "failed to marshal JSON: %w"	13 occurrences	⚠️
Most-duplicated generic wrap: "failed to get current directory: %w"	8 occurrences	⚠️
Most-duplicated generic wrap: "failed to find git root: %w"	8 occurrences	⚠️

Findings

Strengths

• A documented, precise style guide already exists (error-messages skill) with a clear template and a table contrasting "avoid" vs "prefer" phrasing.
• 106 call sites already use the structured NewValidationError helper, showing an established pattern to extend rather than invent.
• pkg/parser is comparatively clean (only 14 files with generic wrappers), suggesting the pattern is enforceable — it's mainly pkg/cli that lags.

Areas for Improvement

• ❌ High severity: 257 generic "failed to X: %w" wrappers with zero recovery guidance, concentrated in pkg/cli (the primary user-facing surface).
• ❌ High severity: 215 bare invalid ... errors that don't state the expected format or give an example, violating the "what's expected"/"example" requirements of the guide.
• ⚠️ Medium severity: High duplication of identical low-context error strings (git root, current directory, home directory, temp directory, JSON marshal) — a shared helper would fix many at once and prevent regressions.
• ⚠️ Medium severity: No automated lint/check exists to prevent regression of the style guide (a custom Go analyzer per .github/skills/go-linters/SKILL.md could catch this going forward).

Detailed Analysis

Representative offending call sites (file:line):

• pkg/cli/outcomes_command.go:196, pkg/cli/deps_report.go:249, pkg/cli/run_workflow_execution.go:612 — fmt.Errorf("failed to marshal JSON: %w", err)
• pkg/gitutil/gitutil.go:134, pkg/cli/trial_repository.go:257,663 — fmt.Errorf("failed to get current directory: %w", err)
• pkg/cli/runner_guard.go:62, pkg/cli/actionlint.go:305, pkg/cli/poutine.go:83 — fmt.Errorf("failed to find git root: %w", err)
• pkg/workflow/repo_config.go:357 — fmt.Errorf("invalid %s: %w", RepoConfigFileName, err) (bare, no expected-format guidance)

Because pkg/gitutil centralizes low-level git helpers used by many pkg/cli commands, adding one-line recovery context there (e.g., "failed to determine current working directory — check that the process has a valid working directory and file-system permissions") would immediately upgrade dozens of downstream call sites without touching each file.

---

🤖 Tasks for Copilot Agent

NOTE TO PLANNER AGENT: Split the following tasks into individual work items.

Improvement Tasks

Task 1: Add recovery guidance to shared gitutil error wrappers

Priority: High
Estimated Effort: Small
Focus Area: Error Message & Diagnostics Quality

Description: Update the shared helpers in pkg/gitutil/gitutil.go (and any equivalent helper in pkg/cli) that wrap "failed to get current directory", "failed to find git root", and "failed to get home directory" errors so each includes a short actionable recovery hint per .github/skills/error-messages/SKILL.md, instead of bare %w wraps. Since these are shared low-level helpers, this single change propagates improved messages to dozens of call sites (8+ occurrences each) across pkg/cli.

Acceptance Criteria:

• pkg/gitutil/gitutil.go:134 and equivalent call sites use a message following [what's wrong]. [what's expected/how to fix] format
• No new duplicate literal error strings are introduced elsewhere (search call sites and route through the shared helper instead of local fmt.Errorf)
• make test-unit passes with no regressions in tests asserting on old error text

Code Region: pkg/gitutil/gitutil.go, pkg/cli/runner_guard.go:62, pkg/cli/actionlint.go:305, pkg/cli/poutine.go:83

In pkg/gitutil/gitutil.go, find the function(s) that return
fmt.Errorf("failed to get current directory: %w", err) and
fmt.Errorf("failed to find git root: %w", err) (or similarly named git-root
lookup helpers). Update these error messages to follow the repo's error
message style guide in .github/skills/error-messages/SKILL.md: state what
went wrong plus a short actionable recovery hint, e.g.
fmt.Errorf("failed to determine current working directory: %w (check that the process has a valid working directory and read permissions)", err)
and
fmt.Errorf("failed to find git repository root: %w (run this command from inside a git repository, or use 'git init')", err).
Then search pkg/cli for other files calling similar patterns (e.g.
pkg/cli/trial_repository.go, pkg/cli/actionlint.go, pkg/cli/poutine.go,
pkg/cli/runner_guard.go) and, where they wrap the same underlying failure
locally instead of calling the shared gitutil helper, route them through the
shared helper so the improved message is reused rather than duplicated. Run
`go build ./...` and `make test-unit` to confirm no regressions, and run
`make fmt` after editing Go files.

---

Task 2: Deduplicate and improve "failed to marshal JSON" error wraps

Priority: Medium
Estimated Effort: Small
Focus Area: Error Message & Diagnostics Quality

Description: 13 call sites across pkg/cli (e.g., outcomes_command.go:196, deps_report.go:249, run_workflow_execution.go:612) use the identical bare wrapper fmt.Errorf("failed to marshal JSON: %w", err). Introduce a small shared helper function (e.g., marshalJSONOrError(v any, context string) ([]byte, error)) in a suitable shared pkg/cli utility file that produces a context-specific, actionable error message, and replace the duplicated call sites with calls to it.

Acceptance Criteria:

• A shared helper exists and is used by at least the 3 files identified (and ideally all 13 occurrences)
• Error messages include which output/context failed to marshal (not just a bare "failed to marshal JSON")
• Existing tests referencing the old error text are updated accordingly
• make test-unit passes

Code Region: pkg/cli/outcomes_command.go:196, pkg/cli/deps_report.go:249, pkg/cli/run_workflow_execution.go:612

Search pkg/cli for all occurrences of the exact string
fmt.Errorf("failed to marshal JSON: %w", err) (13 occurrences reported,
including pkg/cli/outcomes_command.go:196, pkg/cli/deps_report.go:249, and
pkg/cli/run_workflow_execution.go:612). Introduce a small shared helper in an
appropriate existing pkg/cli utility file (do not create a new file if a
suitable one already exists, e.g. a general "helpers" or "utils" file) such
as:

func marshalJSONOrWrap(v any, context string) ([]byte, error) {
    data, err := json.Marshal(v)
    if err != nil {
        return nil, fmt.Errorf("failed to marshal %s to JSON: %w", context, err)
    }
    return data, nil
}

Then replace each of the 13 call sites with a call to this helper, passing a
short descriptive context string (e.g. "outcomes report", "dependency
report", "run execution result") appropriate to that call site, per
.github/skills/error-messages/SKILL.md guidance on actionable, specific error
messages. Update any unit tests asserting on the old literal error string.
Run `make fmt` and `make test-unit` to validate.

---

Task 3: Convert bare invalid %s: %w validation errors in pkg/workflow to NewValidationError

Priority: High
Estimated Effort: Medium
Focus Area: Error Message & Diagnostics Quality

Description: pkg/workflow contains validation error sites such as pkg/workflow/repo_config.go:357 (fmt.Errorf("invalid %s: %w", RepoConfigFileName, err)) that report a bare "invalid" without stating the expected format or an example, violating .github/skills/error-messages/SKILL.md. Since pkg/workflow already has 106 uses of the structured NewValidationError(field, value, reason, suggestion) helper elsewhere in the codebase, migrate the bare invalid %s: %w sites in *_validation.go-style logic (and closely related validation call sites) to use NewValidationError with a concrete suggestion example.

Acceptance Criteria:

• pkg/workflow/repo_config.go:357 and similar bare invalid %s: %w sites in validation-focused files are converted to NewValidationError (or, where NewValidationError doesn't fit because it's a lower-level wrap, given an explicit expected-format/example suffix)
• Each new/updated message states what's wrong, what's expected, and includes an example per the style guide template
• Existing tests are updated for new error text; new/adjusted unit tests confirm the improved message content
• make test-unit passes

Code Region: pkg/workflow/repo_config.go:357 and other *_validation.go files with bare invalid %s: %w patterns

In pkg/workflow, search for fmt.Errorf calls matching the pattern
"invalid %s: %w" or other bare "invalid ..." messages that do not state the
expected format or an example (see .github/skills/error-messages/SKILL.md for
the required template: "[what's wrong]. [what's expected]. [example]").
Start with pkg/workflow/repo_config.go:357 —
fmt.Errorf("invalid %s: %w", RepoConfigFileName, err) — and any other
occurrences located in files whose names end in _validation.go or that
clearly perform user-facing config validation (grep for
'fmt.Errorf("invalid %s: %w"' across pkg/workflow to find all matches).
For each qualifying site, replace the bare fmt.Errorf with the existing
NewValidationError(field, value, reason, suggestion) helper (already used
106 times elsewhere in the codebase — follow its existing call signature and
import it from wherever it's currently defined), providing a concrete
'suggestion' string with a minimal valid YAML/example snippet and a ✓ marker
where ambiguity is likely, matching the style used by other NewValidationError
call sites in the same package. Do not change sites where the error is a
low-level operational wrap unrelated to user config (only touch clear
validation-of-user-input cases). Update any existing tests asserting on the
old error text. Run `make fmt` and `make test-unit` to validate no
regressions.

---

Task 4: Add a custom linter check for the discouraged generic error-wrap pattern

Priority: Medium
Estimated Effort: Medium
Focus Area: Error Message & Diagnostics Quality

Description: To prevent regression of the 257 generic "failed to X: %w" wrappers found in this audit, add a custom Go analyzer (per the existing .github/skills/go-linters/SKILL.md pattern for adding analyzers under pkg/linters) that flags new fmt.Errorf calls whose format string starts with "failed to " and ends with ": %w" with no additional context/recovery text, so future PRs are nudged toward the style guide rather than silently accumulating more of the same pattern.

Acceptance Criteria:

• A new analyzer is added under pkg/linters following the conventions in .github/skills/go-linters/SKILL.md
• The analyzer flags (does not necessarily fail the build for existing 257 occurrences — an allowlist/baseline may be needed) new/added generic "failed to X: %w" patterns without additional context
• Analyzer has unit tests covering both a flagged case and an acceptable case (message including extra context)
• Documented in the linter's own doc comment / relevant skill reference

Code Region: pkg/linters/ (new analyzer file)

Following the pattern documented in .github/skills/go-linters/SKILL.md and
.github/skills/pr-to-go-linter/SKILL.md, add a new custom Go static analyzer
under pkg/linters that detects fmt.Errorf calls whose literal format string
matches the discouraged generic-wrapper pattern described in
.github/skills/error-messages/SKILL.md: a format string starting with
"failed to " and ending with ": %w" with no additional recovery-guidance text
in between beyond a short action description (e.g.
fmt.Errorf("failed to get current directory: %w", err) should be flagged;
fmt.Errorf("failed to find git repository root: %w (run this command from
inside a git repository)", err) should NOT be flagged because it contains
additional guidance).  Implement the analyzer using the go/analysis framework
consistent with existing analyzers in pkg/linters (look at an existing
analyzer file there for the boilerplate: package structure, Analyzer var,
run function using go/ast inspection of ast.CallExpr nodes for fmt.Errorf).
Add unit tests with a small testdata Go file containing both a flagged case
and a passing case, following the existing test conventions in pkg/linters.
Given the audit found 257 existing occurrences, wire the analyzer to be
opt-in or run in report-only/warn mode initially (do not fail existing CI)
so it acts as a guardrail against new occurrences without requiring an
immediate mass rewrite; document this baseline decision in the analyzer's
doc comment. Run `make fmt` and existing linter test commands
(e.g. via `make lint` or the project's Go test invocation for pkg/linters)
to validate.

---

📊 Historical Context

Previous Focus Areas

Date	Focus Area	Type	Custom	Key Outcomes
2026-08-06	Error Message & Diagnostics Quality	Custom	Y	First run — established baseline metrics (257 generic wrappers, 215 bare invalid messages)

---

🎯 Recommendations

Immediate Actions (This Week)

1. Add recovery guidance to shared gitutil error wrappers (Task 1) — Priority: High
2. Convert bare invalid %s: %w validation errors in pkg/workflow to NewValidationError (Task 3) — Priority: High

Short-term Actions (This Month)

1. Deduplicate "failed to marshal JSON" wraps via a shared helper (Task 2) — Priority: Medium
2. Add a custom linter guardrail for the generic-wrapper pattern (Task 4) — Priority: Medium

Long-term Actions (This Quarter)

1. Expand the NewValidationError migration across all of pkg/workflow and pkg/cli validation logic, driving the 215 bare-invalid count toward zero — Priority: Low

---

📈 Success Metrics

• Generic "failed to X: %w" wrapper call sites: 257 → < 100 (targeted reduction via shared helpers + linter guardrail)
• Bare invalid ... messages missing expected/example: 215 → < 100
• NewValidationError usage: 106 → 150+ (increased adoption for config validation)

---

Next Steps

1. Review and prioritise the tasks above
2. Assign tasks to Copilot coding agent via planner agent
3. Track progress on improvement items
4. Re-evaluate this focus area in 1 quarter (or sooner if reused per the 10% reuse rule)

---

Generated by Repository Quality Improvement Agent
Next analysis: 2026-08-07 — Focus area selected by diversity algorithm

Generated by ⚡ Repository Quality Improvement Agent · auto · 28.5 AIC · ⌖ 2.52 AIC · ⊞ 9.5K · ◷

• expires on Aug 7, 2026, 5:27 AM UTC-08:00

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-08-07T13:27:19.318Z.

Closed by Workflow