[repository-quality] 🎯 Repository Quality Improvement Report - Error Message Actionability

[github-actions[bot]]

Analysis Date: 2026-08-21
Focus Area: Error Message Actionability (validation & wrapped errors)
Strategy Type: Custom
Custom Area: Yes — the repo already has a dedicated .github/skills/error-messages/SKILL.md style guide mandating "what's wrong / what's expected / how to fix" errors with examples, but a codebase-wide scan shows hundreds of fmt.Errorf call sites that don't follow it. This is a high-leverage, repo-specific gap rather than a generic category.

Executive Summary

gh-aw maintains an explicit error-message style guide (error-messages skill) requiring every validation error to state what's wrong, what's expected, and provide a concrete example, and to prefer NewValidationError(field, value, reason, suggestion) over bare fmt.Errorf in *_validation.go files. A repository scan found 2,666 fmt.Errorf call sites across pkg/, including roughly 245 "invalid X: %s"-style messages with no suggestion or example, and 509 "failed to X: %w" wrapper errors that pass through a low-level error with no recovery guidance. NewValidationError itself has zero usages in pkg/parser, despite that package containing the bulk of frontmatter/workflow validation logic (import_field_extractor.go, mcp.go, remote_workflow_spec.go, etc.).

This means users hitting common failures (bad repo slug format, bad workflow spec host, bad mcp-servers config) get terse errors with no actionable fix path, increasing support burden and back-and-forth debugging. Bringing these call sites in line with the existing skill is low-risk (pure string/wrapping changes, no logic changes) and immediately improves CLI usability for both human and Copilot-agent-driven workflows.

Recommended next steps: (1) retrofit pkg/repoutil and pkg/parser validation errors with actionable, example-driven messages per the skill template, (2) introduce NewValidationError usage in pkg/parser where currently absent, (3) add a lightweight lint/grep-based CI check to catch new bare "invalid X: %s"-style errors going forward.

Full Analysis Report

Focus Area: Error Message Actionability

Current State Assessment

The repository already codifies error-message expectations in .github/skills/error-messages/SKILL.md:

• Template: [what's wrong]. [what's expected]. [example of correct usage]
• Prefer NewValidationError(field, value, reason, suggestion) in *_validation.go files
• Avoid generic wrappers like fmt.Errorf("failed to X: %w", err) unless recovery guidance is added
• Avoid standalone negative wording (invalid, cannot, must, failed) without pairing it with expected behavior

A scan of pkg/ shows this guidance is inconsistently applied.

Metrics Collected:

Metric	Value	Status
Total fmt.Errorf call sites in pkg/	2,666	⚠️
Bare "invalid ... : %s"-style errors (no example/suggestion)	~245	⚠️
Generic "failed to X: %w" wrapper errors	509	⚠️
NewValidationError usages in pkg/parser	0	❌
error-messages skill exists and is documented	Yes	✅

Findings

Strengths

• A clear, well-written error-message style guide already exists (.github/skills/error-messages/SKILL.md) with a good/bad example table.
• Some call sites already follow the pattern well, e.g. pkg/parser/remote_workflow_spec.go:91 (invalid workflowspec host %q — expected a GitHub host... (for example: ...)) and pkg/parser/mcp.go:93 (includes a full YAML example).
• NewValidationError exists as a helper for structured validation errors (used elsewhere in the codebase for schema/frontmatter validation).

Areas for Improvement

• [Medium] pkg/repoutil/repoutil.go:20 — fmt.Errorf("invalid repo format: %s", slug) gives no example of the expected owner/repo format, contradicting the skill's own documented ✅ example (invalid repo format '%s' — expected 'owner/repo' format (for example: 'github/gh-aw')).
• [Medium] pkg/cli/preconditions.go:170, pkg/cli/fetch.go:95, pkg/cli/outcome_eval.go:518 — repeat the same bare "invalid repo(sitory)... : %s" pattern without guidance, duplicating the same fixable gap across multiple files.
• [Low] pkg/cli/shell_completion.go:209,281 — invalid bashrc path: %s / invalid zshrc path: %s give no reason the path was rejected nor what a valid path looks like.
• [Medium] pkg/parser (frontmatter/workflow parsing, the most user-facing validation surface) has zero NewValidationError usages, meaning its ~31 files with fmt.Errorf rely entirely on ad hoc string formatting instead of the structured helper the skill recommends for *_validation.go logic.
• [Low] 509 "failed to X: %w" wrappers exist repo-wide; not all need fixing (some are fine as pure propagation), but a sampling in pkg/parser/workflow_update.go and pkg/parser/frontmatter_hash.go shows chains of 5+ near-identical wrappers (failed to read file, failed to parse frontmatter, failed to extract frontmatter) with no added context about which file/operation, making log triage harder.

Detailed Analysis

The core risk is drift between documented style and actual code: the skill guide is high quality, but nothing enforces it. Because gh aw is a CLI used interactively and by Copilot coding agents parsing error text to decide next steps, ambiguous errors like invalid repo format: %s cost extra round trips (the agent/user must guess the expected format). Centralizing the fixed strings in pkg/repoutil (used by many CLI commands) would have outsized impact for low effort, since SplitRepoSlug is likely called from multiple command paths.

---

🤖 Tasks for Copilot Agent

NOTE TO PLANNER AGENT: Split the following tasks into individual work items.

Improvement Tasks

Task 1: Make repo-slug validation errors actionable in pkg/repoutil

Priority: High
Estimated Effort: Small
Focus Area: Error Message Actionability

Description: Update SplitRepoSlug in pkg/repoutil/repoutil.go to return an error message matching the skill's documented format: state what's wrong, the expected owner/repo format, and a concrete example.

Acceptance Criteria:

• Error message follows the template [what's wrong]. [what's expected]. [example]
• Message includes an example value (e.g. github/gh-aw)
• Existing tests for SplitRepoSlug still pass; add/update a test asserting the new message content
• make fmt run after the change

Code Region: pkg/repoutil/repoutil.go (function SplitRepoSlug, line ~20)

In pkg/repoutil/repoutil.go, the SplitRepoSlug function returns
`fmt.Errorf("invalid repo format: %s", slug)` when the slug isn't
in `owner/repo` form. Update this error message to follow the
project's error-message style guide (.github/skills/error-messages/SKILL.md):
state what's wrong, what's expected, and give a concrete example, e.g.:

fmt.Errorf("invalid repo format %q — expected 'owner/repo' format (for example: 'github/gh-aw')", slug)

Update or add a unit test in pkg/repoutil that asserts the new error
message content. Run `make fmt` afterward. Do not change the function's
behavior, only the error string.

---

Task 2: Apply consistent actionable messaging to duplicated "invalid repo(sitory)" errors in pkg/cli

Priority: Medium
Estimated Effort: Small
Focus Area: Error Message Actionability

Description: pkg/cli/preconditions.go:170, pkg/cli/fetch.go:95, and pkg/cli/outcome_eval.go:518 each return a bare fmt.Errorf("invalid repo... : %s", repo)/fmt.Errorf("invalid repository slug: %s", ...) without an example or expected format. Align these with the improved message introduced in Task 1.

Acceptance Criteria:

• All three call sites updated to state the expected owner/repo format with an example
• Wording is consistent across the three files (reuse a shared format string or helper if practical)
• Existing tests updated to match new message text; no behavior change
• make fmt run after changes

Code Region: pkg/cli/preconditions.go:170, pkg/cli/fetch.go:95, pkg/cli/outcome_eval.go:518

In pkg/cli/preconditions.go (line ~170), pkg/cli/fetch.go (line ~95),
and pkg/cli/outcome_eval.go (line ~518), there are three separate
fmt.Errorf calls of the form:
  fmt.Errorf("invalid repository format: %s", repoSlug)
  fmt.Errorf("invalid repository slug: %s", spec.RepoSlug)
  fmt.Errorf("invalid repo for root tracing: %s", repo)

Per .github/skills/error-messages/SKILL.md, update each to state what's
wrong, the expected 'owner/repo' format, and an example, e.g.:
  fmt.Errorf("invalid repository format %q — expected 'owner/repo' format (for example: 'github/gh-aw')", repoSlug)

Keep wording consistent across all three. Update any existing tests
that assert on the old error text. Run `make fmt` after editing. Do not
change control flow or behavior — only the error message strings.

---

Task 3: Add actionable guidance to shell_completion.go path validation errors

Priority: Low
Estimated Effort: Small
Focus Area: Error Message Actionability

Description: pkg/cli/shell_completion.go:209 and :281 return invalid bashrc path: %s / invalid zshrc path: %s with no reason or fix guidance. Add context about why the path is invalid (e.g., does not exist, not writable) and what a valid path looks like.

Acceptance Criteria:

• Both error messages explain what's wrong (e.g., path doesn't exist / isn't a regular file) and suggest a fix (e.g., pass --rc-file <path> or check $HOME)
• Existing shell-completion tests updated/pass
• make fmt run after the change

Code Region: pkg/cli/shell_completion.go:209,281

In pkg/cli/shell_completion.go, lines ~209 and ~281 return
fmt.Errorf("invalid bashrc path: %s", bashrcPath) and
fmt.Errorf("invalid zshrc path: %s", zshrcPath) respectively, with no
explanation of why the path failed validation or how to fix it.

Per .github/skills/error-messages/SKILL.md, update both messages to
state what's wrong (e.g., the path does not exist or is not accessible),
what's expected, and a suggested fix, for example:
  fmt.Errorf("bashrc path %q does not exist or is not accessible — verify the path or set $HOME to a valid directory", bashrcPath)

Adjust the actual validation logic only if needed to produce an accurate
reason (e.g., distinguishing "does not exist" vs "not a file"); otherwise
keep behavior unchanged. Update any existing tests asserting old message
text. Run `make fmt` after editing.

---

Task 4: Introduce NewValidationError usage in pkg/parser for at least one high-traffic validation path

Priority: Medium
Estimated Effort: Medium
Focus Area: Error Message Actionability

Description: pkg/parser has zero usages of the NewValidationError helper despite being the primary frontmatter/workflow validation surface (import_field_extractor.go, inline_skill_extractor.go, sub_agent_extractor.go). Convert the duplicate-name validation errors (pkg/parser/inline_skill_extractor.go:118 and pkg/parser/sub_agent_extractor.go:238, both currently fmt.Errorf("duplicate ... name %q", name)) to use NewValidationError(field, value, reason, suggestion) for consistent, structured, example-driven output.

Acceptance Criteria:

• Locate the NewValidationError helper definition (likely in pkg/parser or a shared validation package) and confirm its signature
• Convert both duplicate-name error sites to use NewValidationError with a field naming the config path, a reason describing the duplicate, and a suggestion with an example of renaming/removing the duplicate
• Existing tests for inline skills and sub-agents updated to match new error type/message; add a test if none exists for the duplicate-name case
• make fmt and targeted go test ./pkg/parser/... pass

Code Region: pkg/parser/inline_skill_extractor.go:118, pkg/parser/sub_agent_extractor.go:238

In pkg/parser, two validation error sites do not use the project's
NewValidationError helper, unlike the pattern recommended in
.github/skills/error-messages/SKILL.md:
  pkg/parser/inline_skill_extractor.go:118 -> fmt.Errorf("duplicate inline skill name %q", name)
  pkg/parser/sub_agent_extractor.go:238 -> fmt.Errorf("duplicate inline sub-agent name %q", name)

First, find the NewValidationError(field, value, reason, suggestion)
helper definition in this repo (grep for "func NewValidationError").
Then convert both call sites above to use NewValidationError instead of
fmt.Errorf, providing:
  - field: the relevant frontmatter path (e.g., "skills" or "sub-agents")
  - value: the duplicate name
  - reason: "duplicate name '<name>' is already defined"
  - suggestion: an example showing how to rename one of the duplicates
    or remove it, following the skill's example-driven format

Update or add tests in pkg/parser covering the duplicate-name error
case for both inline skills and sub-agents, asserting on the new error
type/message content. Run `make fmt` and `go test ./pkg/parser/...`
(or the project's targeted test command) to confirm no regressions.

---

📊 Historical Context

Previous Focus Areas

Date	Focus Area	Type	Custom	Key Outcomes
2026-08-21	Error Message Actionability	Custom	Y	First run — established baseline metrics (245 bare "invalid" errors, 509 generic wrappers, 0 NewValidationError uses in pkg/parser)

---

🎯 Recommendations

Immediate Actions (This Week)

1. Fix the four flagged task items (repo-slug and shell-completion error messages) — Priority: High

Short-term Actions (This Month)

1. Audit and convert remaining pkg/parser validation errors to NewValidationError where structured field/value/reason/suggestion applies — Priority: Medium
2. Add a CI grep-based check flagging new fmt.Errorf("invalid ...: %s", ...) patterns lacking "expected"/"example" text — Priority: Medium

Long-term Actions (This Quarter)

1. Sweep the 509 "failed to X: %w" wrapper errors and add operation-specific context (e.g., which file, which import path) where currently absent — Priority: Low

---

📈 Success Metrics

• Bare "invalid X: %s" errors without example/suggestion: ~245 → 0
• NewValidationError usages in pkg/parser: 0 → ≥5 (covering the most common validation failures)

---

Next Steps

1. Review and prioritise the tasks above
2. Assign tasks to Copilot coding agent via planner agent
3. Track progress on improvement items
4. Re-evaluate this focus area in 1 quarter

---

Generated by Repository Quality Improvement Agent
Next analysis: 2026-08-22 — Focus area selected by diversity algorithm

Generated by ⚡ Repository Quality Improvement Agent · auto · 27.3 AIC · ⌖ 11.3 AIC · ⊞ 9.9K · ◷

• expires on Aug 22, 2026, 5:15 AM UTC-08:00

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-08-22T13:15:31.876Z.

Closed by Workflow