[repository-quality] 🎯 Repository Quality Improvement Report - Error Message Quality & User Experience #33768
Closed
Replies: 1 comment
-
|
This discussion was automatically closed because it expired on 2026-05-22T13:58:43.446Z.
|
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Analysis Date: 2026-05-21
Focus Area: Error Message Quality & User Experience
Strategy Type: Custom
Custom Area: Yes — Error messaging is critical for CLI tools; this analysis evaluates consistency, actionability, and adherence to the error-messages skill guidelines.
Executive Summary
gh-aw demonstrates strong error wrapping practices (65.2% of errors use
%wfor proper context propagation) and has established comprehensive error message guidelines in.github/skills/error-messages/SKILL.md. However, analysis reveals significant inconsistencies in user-facing error output patterns:Key Findings:
fmt.Errorfcalls vs only 70 console-formatted error messagesfmt.Println(stdout) instead of stderrconsole.FormatErrorMessage, but many still mix raw error outputImpact: Inconsistent error formatting degrades user experience, makes errors harder to scan in terminal output, and violates the repository's own console formatting standards documented in AGENTS.md.
Full Analysis Report
Focus Area: Error Message Quality & User Experience
Current State Assessment
The repository has established strong foundations:
.github/skills/error-messages/SKILL.mdMetrics Collected:
fmt.Errorfcalls%w)fmt.Printlnin CLI (diagnostic)Findings
Strengths
%wfor proper context propagation.github/skills/error-messages/SKILL.mdprovides clear patternspkg/console/package supports consistent formattingdocker_validation.goshow excellent actionable error messagesAreas for Improvement
High Severity:
Inconsistent error formatting — Only 70 console-formatted errors vs 1,844 raw errors
pkg/workflow/*validation*.go(38,838 LOC) andpkg/cli/Diagnostic output to stdout — 64
fmt.Printlncalls should use stderrpkg/cli/audit_cross_run_render.go,compile_stats.go, etc.Medium Severity:
3. Bare
return errwithout context — ~20 instances in compile/run commandspkg/cli/compile_infrastructure.go:29,59,compile_pipeline.go, etc.Low Severity:
5. Missing error message audit — No systematic check for guideline compliance
Detailed Analysis
1. Console Formatting Adoption Gap
Current state: The repository has excellent console formatting infrastructure:
Problem: Validation code returns raw errors:
Root cause: Validation functions correctly return structured errors, but CLI commands don't consistently apply console formatting before display.
2. Output Stream Violations
Files with incorrect stdout usage (should use stderr):
pkg/cli/audit_cross_run_render.go— 13+ diagnosticfmt.Printlncallspkg/cli/compile_stats.go— Recovery plan output to stdoutpkg/cli/domains_command.go— Status messages to stdoutpkg/cli/forecast.go— Analysis output to stdout (1,142 LOC file)pkg/cli/hash_command.go— Hash output correctly uses stdout ✅Unix convention violation:
3. Error Context Loss
Pattern: Bare
return errloses context:Impact: Error chains become opaque; users can't determine root cause.
4. Error Message Quality Examples
Excellent (follows skill guidelines):
Needs improvement (too generic):
🤖 Tasks for Copilot Agent
NOTE TO PLANNER AGENT: Split the following tasks into individual work items.
Improvement Tasks
Task 1: Audit and Fix CLI Error Formatting
Priority: High
Estimated Effort: Medium
Focus Area: Error Message Quality
Description: Systematically audit all CLI command files to ensure errors are formatted using
console.FormatErrorMessage()before output to stderr. Fix the 37 command files that currently use raw error output.Acceptance Criteria:
fmt.Fprintln(os.Stderr, err)patterns replaced withconsole.FormatErrorMessage(err.Error())fmt.Fprintln(os.Stderr, err)calls inpkg/cli/*_command.gofilesmake lint && make test-unitsuccessfullyCode Region:
pkg/cli/*_command.go, specifically files matchinggrep -l 'fmt\.Fprintln.*os\.Stderr.*err' pkg/cli/*.goPriority files:
pkg/cli/access_log.gopkg/cli/add_command.gopkg/cli/audit.gopkg/cli/compile_*.go(multiple files)Guidelines
Error Formatting Rule (from AGENTS.md):
Output Routing (from AGENTS.md):
Validation
After making changes:
make build && make fmtmake lintand verify zero errorsmake test-unitto ensure tests passExample Fix
Before:
After:
Work systematically through the affected files and create a single PR with all fixes.
Priority files:
pkg/cli/audit_cross_run_render.go— Markdown report headers to stderrpkg/cli/compile_stats.go— Recovery plan output to stderrpkg/cli/domains_command.go— Status messages to stderrpkg/cli/forecast.go— Analysis output to stderr (large file: 1,142 LOC)Guidelines
Diagnostic Output:
Structured Output (keep on stdout):
Validation
After making changes:
make build && make fmt./gh-aw compile workflow.md 2>/dev/nullshould show only structured output./gh-aw compile workflow.md >/dev/nullshould show progress to terminalmake test-unitto ensure tests passExample Fix
Before:
After:
Work systematically through the affected files and create a single PR with all fixes.
[what's wrong]. [what's expected]. [example of correct usage]
After (actionable with example):
Before (missing example):
After (with YAML example):
Validation
After making changes:
make build && make fmtmake lintand verify zero errorsmake test-unit— update test expectations if error messages changedNotes
docker_validation.goas a reference for good error messagesCreate a single PR with improved error messages for the top validation errors.
Priority locations:
pkg/cli/compile_infrastructure.go:29— Line 29: add contextpkg/cli/compile_infrastructure.go:59— Line 59: add contextpkg/cli/compile_pipeline.go— Multiple locations (478, 489, 533, 542, 555, 560, 594)pkg/cli/run.go— Lines 28, 37, 75, 84pkg/cli/run_workflow_execution.go— Multiple locationsGuidelines
Error Wrapping Best Practice:
Context Guidelines:
%wto preserve error chain forerrors.Is()anderrors.As()Validation
After making changes:
make build && make fmtmake lintand verify zero errorsmake test-unitto ensure tests passExample Fix
Before (bare return):
After (with context):
Before (bare return in loop):
After (with context):
Work systematically through the affected files and create a single PR with all fixes.
Inspect AST for patterns:
ast.ReturnStmtwith bareerridentifier inpkg/cli/ast.CallExprtofmt.Fprintlnwithos.Stderrand error argsast.CallExprtofmt.Printlnin CLI files (exclude test files)Report diagnostics:
Add to
pkg/linters/linters.go:Testing
Create
analyzer_test.gowith test cases:Create testdata files:
testdata/src/a/bare_return.go— Test bare return err detectiontestdata/src/a/unformatted_error.go— Test unformatted stderr errorstestdata/src/a/stdout_diagnostic.go— Test stdout diagnostic detectionValidation
After implementation:
go test ./pkg/linters/errormessage/— verify all tests passmake lint— linter should integrate cleanlypkg/linters/README.mdReferences
pkg/linters/workflowcall/,pkg/linters/testbuildtag/.github/skills/error-messages/SKILL.mdpkg/console/console.goCreate a single PR with the new linter, tests, and integration.
Beta Was this translation helpful? Give feedback.
All reactions