Repository Quality Report: Error Context Coverage in Compilation Pipeline (2026-03-25)

[github-actions[bot]]

🎯 Repository Quality Improvement Report - Error Context Coverage

Analysis Date: 2026-03-25
Focus Area: Error Context Coverage in Compilation Pipeline
Strategy Type: Custom
Custom Area: Yes — gh-aw is fundamentally a compiler/transformer. Error messages are the primary user-facing feedback mechanism. When a user's workflow markdown fails to compile, the quality of that error message determines whether they can self-serve the fix or need to file an issue.

Executive Summary

Analysis of error handling patterns across pkg/workflow/, pkg/cli/, pkg/parser/, and cmd/ reveals that the codebase has a well-designed structured error system — WorkflowValidationError, OperationError, ConfigurationError, wrappedCompilerError, etc. — all with Suggestion fields. However, there are 227 bare return err calls (out of ~874 error returns, or ~26%) that propagate errors without adding positional context. Most occur in orchestration functions that delegate to well-typed callees, making this a targeted, high-value improvement rather than a systemic failure.

The test-to-source ratio is excellent (220%), the schema covers all frontmatter fields (49 properties in main_workflow_schema.json), and console formatting is used consistently. The primary gap is in the middle layer of the compilation stack — orchestration functions that catch and forward errors without wrapping them in caller context.

Full Analysis Report

Focus Area: Error Context Coverage in Compilation Pipeline

Current State Assessment

Metrics Collected:

Metric	Value	Status
Total error-creating statements	1,741	✅
Error-wrapped with context (fmt.Errorf("%w"))	930	✅
Bare return err (no context added)	227	⚠️
Error context rate	~74%	⚠️
Structured error types with Suggestion field	4 types	✅
Console-formatted errors (FormatErrorMessage)	1,074	✅
Raw unformatted stderr output	0	✅
Source LOC	144,701	—
Test LOC	320,035	✅
Test-to-source ratio	221%	✅

Findings

Strengths

• Rich structured error hierarchy: WorkflowValidationError, OperationError, ConfigurationError, SharedWorkflowError, GitHubToolsetValidationError, wrappedCompilerError, CompilerError
• All structured error types include a Suggestion field for user guidance
• ErrorCollector with fail-fast and collect-all modes enables multi-error reporting
• formatCompilerErrorWithPosition provides IDE-navigable error output with file/line/column
• isFormattedCompilerError prevents double-wrapping in the compiler pipeline
• Zero instances of unformatted diagnostic output to stdout
• JSON schema covers all 49 frontmatter properties with schema_suggestions.go for schema-guided hints

Areas for Improvement

• [Medium] compiler_orchestrator_workflow.go has 11 bare return err in orchestration functions — these are the highest-traffic compilation paths
• [Medium] mcp_config_validation.go has 4 bare return err calls — MCP config errors are common user friction
• [Medium] tools_validation.go has 3 bare return err calls — tool validation is a frequent compile-time failure
• [Low] codex_mcp.go, schema_validation.go, concurrency_validation.go each have 2+ bare returns
• [Low] cmd/gh-aw/main.go is 860 lines — the CLI entry point has grown beyond the recommended 300-line soft limit

Detailed Analysis

The orchestration pattern in compiler_orchestrator_workflow.go is the most interesting case. Functions like processAdditionalConfigurations and processOnSectionAndFilters are composed entirely of if err := subFunc(...); err != nil { return err } chains. Because each subFunc is expected to emit a wrappedCompilerError, the bare propagation is often intentional. However, when a sub-function doesn't emit a structured error (e.g., JSON marshal/unmarshal, I/O errors), the bare propagation surfaces a raw Go error message to the user.

The distinction between "safe to propagate" and "should wrap" bare returns is the key improvement opportunity.

---

🤖 Tasks for Copilot Agent

NOTE TO PLANNER AGENT: The following tasks are designed for GitHub Copilot coding agent execution. Each section is an independent work item.

Improvement Tasks

---

Task 1: Add Error Context to compiler_orchestrator_workflow.go Bare Returns

Priority: High
Estimated Effort: Small
Focus Area: Error Context Coverage

Description:
compiler_orchestrator_workflow.go contains 11 bare return err statements in two orchestration functions: processAdditionalConfigurations (lines ~709–720) and processOnSectionAndFilters (lines ~831–876). When sub-functions emit raw Go errors (I/O errors, JSON errors), they propagate with no caller context. Wrap bare returns that could receive non-compiler errors with fmt.Errorf("...: %w", err).

Acceptance Criteria:

• All bare return err in processAdditionalConfigurations and processOnSectionAndFilters are either wrapped with fmt.Errorf context or documented with a comment explaining why bare propagation is safe (e.g., // error is already a wrappedCompilerError)
• No regression in existing tests (go test ./pkg/workflow/...)
• make fmt passes with no changes

Code Region: pkg/workflow/compiler_orchestrator_workflow.go

In pkg/workflow/compiler_orchestrator_workflow.go, find all `return err` statements in the functions `processAdditionalConfigurations` and `processOnSectionAndFilters`. For each bare return:

1. Check if the calling function (`extractCacheMemoryConfigFromMap`, `ParseToolsConfig`, `extractRepoMemoryConfig`, etc.) already returns a `*wrappedCompilerError`. If yes, add a comment: `// error is already a wrappedCompilerError, safe to propagate bare`
2. If the callee can return a raw Go error (I/O, JSON, network), wrap it: `return fmt.Errorf("failed to process <description>: %w", err)`

Run `make fmt && go test ./pkg/workflow/...` to validate.

---

Task 2: Add Error Context to mcp_config_validation.go Bare Returns

Priority: High
Estimated Effort: Small
Focus Area: Error Context Coverage

Description:
mcp_config_validation.go has 4 bare return err at lines ~105, 250, 254, 263. MCP server configuration errors are one of the most common sources of compilation failures, and users need context about which MCP server block caused the error.

Acceptance Criteria:

• All bare return err in mcp_config_validation.go are wrapped with context that includes the MCP server name or config key
• Error messages follow the pattern: fmt.Errorf("MCP server %q: %w", serverName, err)
• make fmt && go test ./pkg/workflow/... passes

Code Region: pkg/workflow/mcp_config_validation.go

In pkg/workflow/mcp_config_validation.go, find all `return err` statements. For each one, look at the enclosing function to determine the MCP server name or config key that's being validated. Wrap the error with context:

```go
return fmt.Errorf("MCP server %q configuration: %w", serverName, err)

If the server name is not available at the return site, pass it as a parameter or capture it from the enclosing scope. Run make fmt && go test ./pkg/workflow/... to validate.

---

#### Task 3: Add Error Context to `tools_validation.go` Bare Returns

**Priority**: Medium
**Estimated Effort**: Small
**Focus Area**: Error Context Coverage

**Description:**
`tools_validation.go` has 3 bare `return err` at lines ~100, 167, 183. Tool validation errors during compilation are common, and users need to know which tool configuration caused the failure (e.g., `github`, `playwright`, `bash`).

**Acceptance Criteria:**
- [ ] All bare `return err` in `tools_validation.go` are wrapped with the tool name context
- [ ] Error messages follow the pattern: `fmt.Errorf("tools.%s validation: %w", toolName, err)`
- [ ] `make fmt && go test ./pkg/workflow/...` passes

**Code Region:** `pkg/workflow/tools_validation.go`

```markdown
In pkg/workflow/tools_validation.go, locate the three `return err` statements (around lines 100, 167, 183). Examine the enclosing function context to identify which tool is being validated. Wrap the error with the tool name:

```go
return fmt.Errorf("tools.github validation: %w", err)

Use the appropriate tool name for each return site. Run make fmt && go test ./pkg/workflow/... to validate.

---

#### Task 4: Document `return err` Propagation Safety in Orchestrator Functions

**Priority**: Medium
**Estimated Effort**: Small
**Focus Area**: Error Context Coverage

**Description:**
The compilation orchestration pattern (a chain of `if err := subFunc(); err != nil { return err }`) is intentional but undocumented. Future contributors may not know when bare propagation is safe vs when wrapping is required. Add a short code comment at the top of orchestration functions that explains the convention.

**Acceptance Criteria:**
- [ ] Functions that exclusively call `formatCompilerError`-producing sub-functions have a comment: `// Errors from sub-functions are already wrappedCompilerErrors; bare propagation is safe.`
- [ ] At least `processOnSectionAndFilters` and `processAdditionalConfigurations` are documented
- [ ] `make fmt` passes

**Code Region:** `pkg/workflow/compiler_orchestrator_workflow.go`

```markdown
In pkg/workflow/compiler_orchestrator_workflow.go, add a one-line comment at the top of the following functions to document the error propagation contract:

- `processAdditionalConfigurations`
- `processOnSectionAndFilters`

The comment should read:
```go
// Errors from sub-functions are already wrappedCompilerErrors; bare propagation preserves the formatted output.

If any sub-function can return a raw (non-compiler) error, note it inline. Run make fmt to validate.

---

## 📊 Historical Context

<details>
<summary><b>Previous Focus Areas</b></summary>

| Date | Focus Area | Type | Custom | Key Outcomes |
|------|------------|------|--------|--------------|
| 2026-03-25 | Error Context Coverage | Custom | Yes | First run; identified 227 bare returns, 4 actionable tasks |

</details>

---

## 🎯 Recommendations

### Immediate Actions (This Week)
1. **Wrap MCP config bare returns with server name context** — Priority: High (directly improves user error messages for most common failure type)
2. **Wrap tools_validation bare returns with tool name** — Priority: High (targeted, low-risk change)

### Short-term Actions (This Month)
1. **Document orchestrator error propagation contract** — Priority: Medium (prevents future regressions)
2. **Add a linter rule or grep CI check** for bare `return err` in `pkg/workflow/` — Priority: Medium

### Long-term Actions (This Quarter)
1. **Gradually wrap remaining ~150 bare returns** across `pkg/parser/` and `pkg/cli/` — Priority: Low (many are truly safe pass-throughs in callee-structured contexts)

---

## 📈 Success Metrics

Track these metrics to measure improvement in **Error Context Coverage**:

- **Bare `return err` rate**: 227 → target < 100 (focus on `pkg/workflow/` first)
- **Error context rate**: 74% → target > 85%
- **MCP/tools validation bare returns**: 7 → target 0

---

## Next Steps

1. Review and prioritize the 4 tasks above
2. Assign to Copilot coding agent via planner agent (Tasks 1 and 2 are highest impact)
3. Run `grep -rn "return err$" pkg/workflow/ | wc -l` after each task to track progress
4. Re-evaluate this focus area in ~2 weeks after tasks complete

---

**References:**
- [§23543587870](https://github.com/github/gh-aw/actions/runs/23543587870)


> AI generated by [Repository Quality Improvement Agent](https://github.com/github/gh-aw/actions/runs/23543587870) · [history](https://github.com/search?q=repo%3Agithub%2Fgh-aw+%22gh-aw-workflow-call-id%3A+github%2Fgh-aw%2Frepository-quality-improver%22&type=discussions)
> - [x] expires <!-- gh-aw-expires: 2026-03-26T13:37:17.415Z --> on Mar 26, 2026, 1:37 PM UTC

<!-- gh-aw-workflow-id: repository-quality-improver -->
<!-- gh-aw-workflow-call-id: github/gh-aw/repository-quality-improver -->

[github-actions[bot]]

💥 WHOOSH! 🦸

The smoke test agent swoops in from the shadows of the CI pipeline!

KA-POW! 💥 Agent Claude has arrived and the smoke tests are NOMINAL!

"With great automation comes great responsibility..." — Claude, probably

⚡ ZZZAP! All systems green. The agentic workflows live to automate another day!

— Smoke Test Agent, Run 23548639645 🤖

💥 [THE END] — Illustrated by Smoke Claude · ◷