[repository-quality] Repository Quality Improvement Report - Large File Decomposition

[github-actions[bot]]

🎯 Repository Quality Improvement Report - Large File Decomposition

Analysis Date: 2026-05-12
Focus Area: Large File Decomposition
Strategy Type: Custom
Custom Area: Yes — the repository's own AGENTS.md documents a hard limit of 300 lines per file and a recommended 100–200 lines per validator, yet 211 source files exceed 300 lines and 97 exceed 500 lines, making this the most actionable gap.

Executive Summary

gh-aw has grown a class of source files that significantly exceeds its own documented complexity guidelines. Of 372 non-test Go files in pkg/, 211 (57%) are above the 300-line hard limit, and 97 (26%) exceed 500 lines. The most critical offender is a single 508-line method — extractAllImportFields in pkg/parser/import_field_extractor.go — that handles YAML parsing, validation, schema defaults, input substitution, and observability extraction all in one function. Similarly, pkg/workflow/domains.go bundles per-engine domain data as inline var blocks alongside the logic that assembles them, making adding or auditing a new engine error-prone.

These oversized files create concrete costs: slower code review, test coverage gaps (a 508-line function with a single test exercises very little branching), and higher cognitive load for contributors. Decomposition is low-risk here because the repository has an exceptionally strong test suite (2.1× test-to-source LOC ratio) and thorough make agent-finish validation, meaning refactored units can be validated confidently.

The four tasks below are independently actionable, sized from a 30-minute extraction to a 2-hour careful split. All targets were chosen for clear internal seams rather than arbitrary line counts.

Full Analysis Report

Focus Area: Large File Decomposition

Current State Assessment

Metrics Collected:

Metric	Value	Status
Source files >300 lines (AGENTS.md hard limit)	211 / 372 (57%)	❌
Source files >500 lines	97 / 372 (26%)	❌
Largest single file	pkg/parser/import_field_extractor.go — 1127 lines	❌
Largest single function	extractAllImportFields — 508 lines	❌
TODO/FIXME count (all packages)	14	✅
Test-to-source LOC ratio	2.1×	✅

Top 10 oversized files:

File	Lines	Functions	Notes
pkg/parser/import_field_extractor.go	1127	12	1 function is 508 lines
pkg/workflow/frontmatter_extraction_yaml.go	1114	13	commentOutProcessedFieldsInOnSection spans lines 107–688 (581 lines)
pkg/workflow/compiler_yaml_main_job.go	1089	15	Mixed step-generation concerns
pkg/cli/forecast.go	1058	25	Data types, logic, and rendering mixed
pkg/cli/audit.go	1043	11	Audit logic with rendering interleaved
pkg/workflow/domains.go	1040	25	18 engine-specific var blocks + logic
pkg/workflow/compiler_jobs.go	1022	—	Multiple job compilation concerns
pkg/workflow/cache.go	1005	—	Cache logic
pkg/cli/logs_orchestrator.go	995	—	Log orchestration
pkg/workflow/threat_detection.go	984	—	Threat detection

Findings

Strengths

• Excellent test-to-source ratio (2.1×) means refactoring is low-risk
• make agent-finish provides a fast, reliable validation gate
• AGENTS.md already documents the 100–300 line target — intention is clear
• TODO/FIXME count is very low (14), indicating disciplined debt management

Areas for Improvement

• CRITICAL: extractAllImportFields (508 lines) is a god-method with at least 6 distinct responsibilities: frontmatter parsing, tool/MCP merging, engine detection, input schema validation, input substitution, and observability extraction
• CRITICAL: commentOutProcessedFieldsInOnSection in frontmatter_extraction_yaml.go (581 lines) contains deeply nested YAML string manipulation that should be broken into named stages
• HIGH: domains.go inlines engine domain data as var blocks mixed with domain-assembly logic; the data and logic are distinct concerns
• MEDIUM: forecast.go mixes struct definitions, business logic (forecastWorkflow), metadata loading, variant computation, and three rendering formats in one file

Detailed Analysis

extractAllImportFields (508 lines) — The function has identifiable phases, each suitable for extraction:

1. Frontmatter parsing (lines ~101–160) → parseImportFrontmatter()
2. Tool/MCP merging (lines ~160–280) → mergeImportTools()
3. Engine/bot/label accumulation (lines ~280–420) → accumulateImportMetadata()
4. Input schema validation and defaults (lines ~420–530) → already partially in validateWithImportSchema() but inlined here too
5. Observability extraction (lines ~530–609) → extractImportObservability()

domains.go — 18 named var blocks (CopilotDefaultDomains, ClaudeDefaultDomains, etc.) are data, not logic. Moving them to a domains_data.go file and keeping only the assembly functions in domains.go would halve both files and make engine-specific domain lists easy to find and audit.

forecast.go — Natural split into: forecast_types.go (struct definitions), forecast_run.go (core logic), forecast_render.go (table/JSON rendering). This mirrors patterns already used elsewhere in pkg/cli/ (e.g., audit.go / audit_diff.go / audit_report.go).

---

🤖 Tasks for Copilot Agent

NOTE TO PLANNER AGENT: Split the following tasks into individual work items.

Improvement Tasks

Task 1: Extract phases from extractAllImportFields into helper methods

Priority: High
Estimated Effort: Medium
Focus Area: Large File Decomposition

Description: The extractAllImportFields method in pkg/parser/import_field_extractor.go is 508 lines long. Extract each logical phase into a private helper method on importAccumulator: tool/MCP merging, engine/bot/label accumulation, and observability extraction. Each helper should have its own unit test assertions in the existing test file.

Acceptance Criteria:

• extractAllImportFields is reduced to ≤150 lines (orchestration only)
• At least 3 new private methods extracted (mergeImportTools, accumulateImportMetadata, extractImportObservability)
• All existing pkg/parser tests pass (go test ./pkg/parser/)
• No new exported symbols added (pure internal refactor)
• make agent-finish passes

Code Region: pkg/parser/import_field_extractor.go

Refactor `extractAllImportFields` in `pkg/parser/import_field_extractor.go`.

The method is 508 lines (lines 101–609). It has 5 identifiable phases:
1. Frontmatter parsing (~lines 101–160)
2. Tool/MCP merging (~lines 160–280)
3. Engine/bot/label accumulation (~lines 280–420)
4. Input schema validation (already delegated but some logic is inlined)
5. Observability extraction (~lines 530–609)

Extract phases 2, 3, and 5 into private receiver methods on `importAccumulator` named `mergeImportTools`, `accumulateImportMetadata`, and `extractImportObservability`. The main method should become an orchestrator calling these helpers.

Acceptance:
- `extractAllImportFields` ≤150 lines
- All existing tests pass: `go test ./pkg/parser/`
- `make build && make fmt` passes
- No exported API changes

---

Task 2: Split domains.go into data file and logic file

Priority: High
Estimated Effort: Small
Focus Area: Large File Decomposition

Description: pkg/workflow/domains.go (1040 lines) contains 18 engine-specific domain var blocks alongside the assembly and validation logic. Separate the data into domains_data.go and keep only functions in domains.go. This is a mechanical move with zero logic changes.

Acceptance Criteria:

• New file pkg/workflow/domains_data.go created with all var domain declarations
• pkg/workflow/domains.go contains only functions (≤400 lines)
• All pkg/workflow tests pass: go test ./pkg/workflow/
• make agent-finish passes

Code Region: pkg/workflow/domains.go

Split `pkg/workflow/domains.go` (1040 lines) by moving all `var` domain declaration blocks to a new file `pkg/workflow/domains_data.go`. 

The variables to move are all top-level `var` blocks that are pure data (slice and map literals): `CopilotDefaultDomains`, `CodexDefaultDomains`, `ClaudeDefaultDomains`, `GeminiDefaultDomains`, `PiBaseDefaultDomains`, `piProviderDomains`, `PiDefaultDomains`, `CrushBaseDefaultDomains`, `crushProviderDomains`, `CrushDefaultDomains`, `OpenCodeBaseDefaultDomains`, `openCodeProviderDomains`, `OpenCodeDefaultDomains`, `PlaywrightDomains`, `compoundEcosystems`, `runtimeToEcosystem`, `ecosystemPriority`, `engineDefaultDomains`.

Keep in `domains.go`: the logger var, the `//go:embed` directive and byte/map vars for ecosystem JSON, all functions, and the `init()` function.

No logic changes — this is a pure mechanical move.

Acceptance:
- `domains_data.go` exists with only `var` blocks
- `domains.go` ≤450 lines, functions only
- `go test ./pkg/workflow/` passes
- `make build && make fmt` passes

---

Task 3: Split pkg/cli/forecast.go into types, run, and render files

Priority: Medium
Estimated Effort: Medium
Focus Area: Large File Decomposition

Description: pkg/cli/forecast.go (1058 lines, 25 functions) mixes struct type definitions, business logic, metadata helpers, and two rendering functions. Split following the pattern already established by audit.go/audit_report.go/audit_diff.go.

Acceptance Criteria:

• forecast_types.go created with all struct and type definitions
• forecast_render.go created with renderForecastJSON, renderForecastTable, printEpisodeBreakdown, printEvalBreakdown, printVariantBreakdown
• forecast.go contains only command setup and core logic functions (≤400 lines)
• go test ./pkg/cli/ -run ".*[Ff]orecast" passes
• make agent-finish passes

Code Region: pkg/cli/forecast.go

Split `pkg/cli/forecast.go` (1058 lines) into three files following the pattern of `audit.go`/`audit_report.go`/`audit_diff.go`:

1. `pkg/cli/forecast_types.go` — move all struct/type definitions: `ForecastEpisodeSummary`, `ForecastWorkflowResult`, `ForecastVariantResult`, `ForecastEvaluation`, `ForecastResult`, `workflowMeta`, `forecastTableRow`. Keep package-level `var` declarations (`forecastRunLog`, `forecastPeriodDays`) in `forecast.go`.

2. `pkg/cli/forecast_render.go` — move all rendering/output functions: `renderForecastJSON`, `renderForecastTable`, `printEpisodeBreakdown`, `printEvalBreakdown`, `printVariantBreakdown`.

3. `pkg/cli/forecast.go` — keep `NewForecastCommand`, `RunForecast`, and all non-rendering business logic functions.

No logic changes — pure mechanical move.

Acceptance:
- Three files exist as described above
- `go test ./pkg/cli/ -run ".*[Ff]orecast"` passes
- `make build && make fmt` passes

---

Task 4: Add AGENTS.md guideline enforcement note to commentOutProcessedFieldsInOnSection

Priority: Low
Estimated Effort: Small
Focus Area: Large File Decomposition

Description: commentOutProcessedFieldsInOnSection in pkg/workflow/frontmatter_extraction_yaml.go is 581 lines (lines 107–688) — the largest single function in the codebase. It processes YAML string manipulation through deeply nested branching. Add a // TODO(decomp): comment at the top of the function outlining the 4 identifiable stages to guide a future decomposition, and open a follow-up issue in the repository to track it. This serves as a low-cost breadcrumb so the function doesn't keep getting extended without visibility.

Acceptance Criteria:

• A // TODO(decomp): block comment added at line 107 listing the 4 stages: (1) field classification, (2) on-section rewriting, (3) inline comment injection, (4) whitespace normalisation
• make build && make fmt passes
• No logic changes

Code Region: pkg/workflow/frontmatter_extraction_yaml.go:107

Add a structured decomposition guide comment to the top of `commentOutProcessedFieldsInOnSection` in `pkg/workflow/frontmatter_extraction_yaml.go` (line 107).

The function is 581 lines. Without touching logic, add a comment block immediately after the function signature like:

```go
// commentOutProcessedFieldsInOnSection rewrites the on: YAML section so that
// fields already processed by the compiler are commented out.
//
// TODO(decomp): This function has grown to 581 lines. Future decomposition should
// split it into four stages:
//   1. classifyOnSectionFields — determine which fields are compiler-handled vs pass-through
//   2. rewriteOnSectionYAML — apply comment-out transformations field by field
//   3. injectInlineComments — add explanatory inline comments to the output
//   4. normaliseWhitespace — post-process indentation and blank-line consistency

No logic changes. Run make build && make fmt to verify.

---

## 📊 Historical Context

<details>
<summary>Previous Focus Areas</summary>

| Date | Focus Area | Type | Custom | Key Outcomes |
|------|------------|------|--------|--------------|
| 2026-05-12 | Large File Decomposition | Custom | Y | First run — baseline established |

</details>

---

## 🎯 Recommendations

### Immediate Actions (This Week)
1. **Task 2: Split `domains.go`** — Priority: High (mechanical, <1 hour, zero risk)
2. **Task 1: Extract from `extractAllImportFields`** — Priority: High (highest complexity reduction)

### Short-term Actions (This Month)
1. **Task 3: Split `forecast.go`** — Priority: Medium (improves discoverability)

### Long-term Actions (This Quarter)
1. **Task 4: Document `commentOutProcessedFieldsInOnSection`** — Priority: Low (breadcrumb for future decomp)
2. Establish a CI lint rule (`wc -l` or `gocognit`) to flag files growing past 300 lines

---

## 📈 Success Metrics

- **Files >300 lines**: 211 → <150 (target after these 4 tasks: ~208, then incremental)
- **Files >500 lines**: 97 → <90
- **Longest function**: 581 lines → ≤150 lines (Task 1 target)
- **`domains.go` size**: 1040 lines → ≤450 lines (Task 2 target)

---

## Next Steps

1. Review and prioritise the tasks above
2. Assign tasks to Copilot coding agent via planner agent
3. Track progress on improvement items
4. Re-evaluate Large File Decomposition in 30 days

---

*Generated by Repository Quality Improvement Agent*
*Next analysis: 2026-05-13 — Focus area selected by diversity algorithm*




> Generated by [Repository Quality Improvement Agent](https://github.com/github/gh-aw/actions/runs/25737363079) · ● 6.4M · [◷](https://github.com/search?q=repo%3Agithub%2Fgh-aw+%22gh-aw-workflow-call-id%3A+github%2Fgh-aw%2Frepository-quality-improver%22&type=discussions)
> - [x] expires <!-- gh-aw-expires: 2026-05-13T13:30:45.818Z --> on May 13, 2026, 1:30 PM UTC

<!-- gh-aw-agentic-workflow: Repository Quality Improvement Agent, engine: copilot, version: 1.0.43, model: claude-sonnet-4.6, id: 25737363079, workflow_id: repository-quality-improver, run: https://github.com/github/gh-aw/actions/runs/25737363079 -->

<!-- gh-aw-workflow-id: repository-quality-improver -->
<!-- gh-aw-workflow-call-id: github/gh-aw/repository-quality-improver -->

[github-actions[bot]]

This discussion has been marked as outdated by Repository Quality Improvement Agent.

A newer discussion is available at Discussion #31945.