[repository-quality] 🎯 Repository Quality Improvement Report - Test File Bloat & Organization

[github-actions[bot]]

Analysis Date: 2026-08-18
Focus Area: Test File Bloat (custom, refining the "Code Organization" standard category)
Strategy Type: Custom
Custom Area: Yes — the repo's own developer-code-organization skill mandates files of 100–500 lines (800+ only for "core infrastructure"), yet 30 test files exceed 1,500 lines and 17 exceed 2,000 lines, with compiler_jobs_test.go at 4,511 lines. This is a self-inflicted violation of the project's own documented convention, worth surfacing explicitly rather than a generic "Testing" pass.

Executive Summary

gh-aw has an unusually high test-to-source ratio (539,581 test LOC vs 275,701 source LOC, ~2:1), which is generally healthy, but a large share of that test code has accumulated into a small number of monolithic files. Eight test files exceed 2,400 lines each, led by pkg/workflow/compiler_jobs_test.go (4,511 lines / 84 test functions), compiler_safe_outputs_config_test.go (3,836 lines / 48 funcs), and threat_detection_test.go (3,420 lines / 61 funcs). These sizes far exceed the repo's own file-size guideline ("Large files: 500-800 lines... Very large files: 800+ core infrastructure only"), and none of this is core infrastructure — it's flat test-case enumeration that could be split along natural feature/scenario boundaries the same way frontmatter.go was previously refactored (documented as a case study in the developer-code-organization skill).

Splitting these files by functional area (e.g. by job type, safe-output kind, or threat category) would reduce merge-conflict risk on high-traffic files, speed up editor/LSP responsiveness, and make it far easier for both humans and Copilot coding agents to locate the right test when fixing a bug. Non-test source files are in comparatively good shape (only 15 files over 1,000 lines, and most of those — compiler.go, workflow_builder.go, etc. — are legitimately core infrastructure), so this report focuses tasks on the highest-impact test-file splits.

Full Analysis Report

Focus Area: Test File Bloat & Organization

Current State Assessment

Ran line-count and function-count analysis across all *_test.go files (excluding vendor) plus a comparison against the repo's own file-size guidance in .github/skills/developer-code-organization/SKILL.md.

Metrics Collected:

Metric	Value	Status
Test files > 1,500 lines	30	⚠️
Test files > 2,000 lines	17	⚠️
Largest test file	pkg/workflow/compiler_jobs_test.go (4,511 lines, 84 funcs)	❌
Test LOC / Source LOC ratio	539,581 / 275,701 (≈1.96:1)	✅
Non-test source files > 1,000 lines	15	✅ (mostly justified core infra)
TODO/FIXME count (repo-wide)	37	✅
README.md size	439 lines	✅

Findings

Strengths

• Test coverage volume is strong relative to source code (nearly 2:1 LOC ratio).
• Non-test source files are largely within or close to the documented guideline; the 15 files over 1,000 lines are almost all legitimate "core infrastructure" (compiler, CLI command orchestration, config).
• The repo already has a documented precedent and mental model for splitting oversized files (frontmatter.go → 5 focused files), which can be directly reused for test files.

Areas for Improvement

• ❌ High severity: pkg/workflow/compiler_jobs_test.go (4,511 lines / 84 test funcs) is 5.6× the "large file" ceiling (800 lines) and has no sub-file organization by job type.
• ❌ High severity: pkg/workflow/compiler_safe_outputs_config_test.go (3,836 lines / 48 funcs) mixes config validation tests for many unrelated safe-output types in one file.
• ⚠️ Medium severity: pkg/workflow/threat_detection_test.go (3,420 lines / 61 funcs), pkg/workflow/copilot_engine_test.go (3,311 lines / 57 funcs), and pkg/workflow/maintenance_workflow_test.go (3,076 lines / 31 funcs) all show the same monolithic pattern.
• ⚠️ Medium severity: pkg/parser/schema_test.go (2,892 lines), pkg/cli/remote_workflow_test.go (2,707 lines), and pkg/workflow/safe_outputs_import_test.go (2,411 lines) are close behind and will keep growing without a splitting convention.
• ⚠️ Low severity, process gap: no documented naming convention exists yet for split test files (e.g. feature_a_test.go, feature_b_test.go), so future contributors may default back to appending to the existing large file rather than creating a new one.

Detailed Analysis

Using wc -l and grep -c '^func Test' across all *_test.go files (vendor excluded), the eight largest test files account for over 26,000 lines and 393 test functions combined — roughly 5% of the repo's total test LOC concentrated in 8 files out of thousands. Each of these files groups tests by broad subsystem (all compiler job tests, all safe-outputs config tests, all threat-detection tests) rather than by narrower feature boundary, which is exactly the anti-pattern the repo's own developer-code-organization skill warns against ("Avoid: models.go / logic.go / tests.go — all business logic in one file"). Splitting along sub-feature boundaries (e.g., one file per job type within compiler_jobs_test.go, one file per safe-output kind within compiler_safe_outputs_config_test.go) mirrors the existing "Create Functions Pattern" already used for non-test code (create_issue.go, create_pull_request.go, etc.) and would bring these files back under the 800-line guideline without changing any test semantics or reducing coverage.

---

🤖 Tasks for Copilot Agent

NOTE TO PLANNER AGENT: Split the following tasks into individual work items.

Improvement Tasks

Task 1: Split compiler_jobs_test.go by job type

Priority: High
Estimated Effort: Large
Focus Area: Test File Bloat

Description: pkg/workflow/compiler_jobs_test.go is 4,511 lines with 84 test functions covering many distinct job types (activation jobs, safe-output jobs, pre-activation jobs, etc.). Split it into multiple files grouped by job type/feature, following the naming convention compiler_jobs_<subfeature>_test.go, mirroring the existing "Create Functions Pattern" used for non-test files.

Acceptance Criteria:

• No single resulting file exceeds ~800 lines
• All 84 existing test functions are preserved with identical names and behavior
• go test ./pkg/workflow/... passes with no regressions
• Shared test helpers/fixtures extracted to a common compiler_jobs_test_helpers.go if duplicated across new files

Code Region: pkg/workflow/compiler_jobs_test.go

Split pkg/workflow/compiler_jobs_test.go (4,511 lines, 84 test functions) in the gh-aw repo into multiple smaller test files grouped by the job type or feature each test function covers (e.g., activation jobs, pre-activation jobs, safe-output jobs, custom jobs). Follow the naming convention compiler_jobs_<subfeature>_test.go. Keep all existing test function names and assertions identical — this is a pure file-organization refactor, not a behavior change. Extract any shared test helpers or fixtures used across multiple groups into a shared compiler_jobs_test_helpers.go file. Ensure `go test ./pkg/workflow/...` passes after the split with the same test count and same pass/fail results as before. Aim for no resulting file exceeding roughly 800 lines, consistent with the repository's documented file-size guidelines in .github/skills/developer-code-organization/SKILL.md.

---

Task 2: Split compiler_safe_outputs_config_test.go by safe-output type

Priority: High
Estimated Effort: Large
Focus Area: Test File Bloat

Description: pkg/workflow/compiler_safe_outputs_config_test.go (3,836 lines, 48 test functions) validates configuration for many different safe-output types in a single file. Split by safe-output kind (e.g., create-issue config tests, create-pull-request config tests, add-comment config tests) into separate files.

Acceptance Criteria:

• No single resulting file exceeds ~800 lines
• All 48 existing test functions preserved with identical behavior
• go test ./pkg/workflow/... passes with no regressions
• New file names follow compiler_safe_outputs_config_<type>_test.go convention

Code Region: pkg/workflow/compiler_safe_outputs_config_test.go

Split pkg/workflow/compiler_safe_outputs_config_test.go (3,836 lines, 48 test functions) in the gh-aw repo into multiple smaller test files, one group per safe-output type being tested (e.g., create-issue, create-pull-request, add-comment, create-discussion, etc.). Use the naming convention compiler_safe_outputs_config_<type>_test.go. Preserve all existing test function names and assertions exactly — no behavior changes, purely file reorganization. Extract shared setup/fixture code into a common helper file if it's duplicated. Verify `go test ./pkg/workflow/...` passes afterward with the same results as before the split. Target no single file exceeding roughly 800 lines per the repository's file-size guidelines documented in .github/skills/developer-code-organization/SKILL.md.

---

Task 3: Split threat_detection_test.go by detection category

Priority: Medium
Estimated Effort: Medium
Focus Area: Test File Bloat

Description: pkg/workflow/threat_detection_test.go (3,420 lines, 61 test functions) covers many distinct threat/detection categories. Split into files by detection category (e.g., prompt injection detection, secret leakage detection, malicious tool-call detection).

Acceptance Criteria:

• No single resulting file exceeds ~800 lines
• All 61 existing test functions preserved with identical behavior
• go test ./pkg/workflow/... passes with no regressions

Code Region: pkg/workflow/threat_detection_test.go

Split pkg/workflow/threat_detection_test.go (3,420 lines, 61 test functions) in the gh-aw repo into multiple smaller test files grouped by threat/detection category (e.g., prompt injection detection, secret exposure detection, malicious tool usage detection — inspect the existing test function names/comments to determine natural groupings). Use a naming convention like threat_detection_<category>_test.go. Preserve all existing test function names and assertions exactly, with no behavior changes. Extract shared fixtures/helpers into a common file if needed. Verify `go test ./pkg/workflow/...` passes with identical results afterward. Aim for no resulting file exceeding roughly 800 lines per .github/skills/developer-code-organization/SKILL.md.

---

Task 4: Split copilot_engine_test.go and maintenance_workflow_test.go

Priority: Medium
Estimated Effort: Medium
Focus Area: Test File Bloat

Description: pkg/workflow/copilot_engine_test.go (3,311 lines, 57 funcs) and pkg/workflow/maintenance_workflow_test.go (3,076 lines, 31 funcs) are both well over the "very large" 800-line threshold reserved for core infrastructure only — these are test files, not infra. Split each by sub-feature (e.g., copilot engine: prompt building, tool config, MCP wiring; maintenance workflow: scheduling, cleanup, reporting).

Acceptance Criteria:

• Both resulting file sets stay under ~800 lines per file
• All existing test functions preserved with identical behavior in both files
• go test ./pkg/workflow/... passes with no regressions

Code Region: pkg/workflow/copilot_engine_test.go, pkg/workflow/maintenance_workflow_test.go

Split both pkg/workflow/copilot_engine_test.go (3,311 lines, 57 test functions) and pkg/workflow/maintenance_workflow_test.go (3,076 lines, 31 test functions) in the gh-aw repo into smaller test files grouped by sub-feature. For copilot_engine_test.go, group by aspects like prompt/config building, tool configuration, and MCP server wiring. For maintenance_workflow_test.go, group by aspects like scheduling, cleanup logic, and reporting. Use consistent naming conventions such as copilot_engine_<subfeature>_test.go and maintenance_workflow_<subfeature>_test.go. Preserve all existing test function names and assertions exactly — pure reorganization, no behavior changes. Verify `go test ./pkg/workflow/...` passes afterward with identical results. Target no single file exceeding roughly 800 lines, consistent with .github/skills/developer-code-organization/SKILL.md.

---

Task 5: Document a test-file-splitting convention in the code organization skill

Priority: Low
Estimated Effort: Small
Focus Area: Test File Bloat / Process

Description: Add explicit guidance to .github/skills/developer-code-organization/SKILL.md describing when and how to split large *_test.go files (naming convention, shared-helper extraction pattern), so future contributors have a documented pattern to follow instead of appending to already-oversized files.

Acceptance Criteria:

• New section added to the skill documenting the <feature>_<subfeature>_test.go naming convention
• Includes guidance on extracting shared test helpers to <feature>_test_helpers.go
• References the case studies produced by Tasks 1–4 as concrete examples

Code Region: .github/skills/developer-code-organization/SKILL.md

In the gh-aw repo, add a new subsection to .github/skills/developer-code-organization/SKILL.md titled "Splitting Large Test Files" (placed near the existing "Case Study: Refactoring Large Files" section). Document: (1) the trigger threshold (test file > ~800-1000 lines or containing tests for clearly distinct sub-features), (2) the naming convention <feature>_<subfeature>_test.go for the resulting files, (3) the pattern of extracting shared fixtures/helpers into a <feature>_test_helpers.go file when duplicated across the split files, and (4) a short table of real examples from this refactor (e.g., compiler_jobs_test.go split into compiler_jobs_<subfeature>_test.go). Keep the section concise and consistent with the existing documentation style and Mermaid diagram conventions already used elsewhere in the file.

---

📊 Historical Context

Previous Focus Areas

Date	Focus Area	Type	Custom	Key Outcomes
2026-08-18	Test File Bloat & Organization	Custom	Y	First run — established baseline; 30 test files >1,500 lines identified

---

🎯 Recommendations

Immediate Actions (This Week)

1. Split compiler_jobs_test.go and compiler_safe_outputs_config_test.go (the two largest offenders) — Priority: High

Short-term Actions (This Month)

1. Split threat_detection_test.go, copilot_engine_test.go, and maintenance_workflow_test.go — Priority: Medium
2. Document the test-file-splitting convention so the pattern doesn't need to be rediscovered — Priority: Medium

Long-term Actions (This Quarter)

1. Periodically re-run this analysis to catch newly-grown test files before they reach 2,000+ lines — Priority: Low

---

📈 Success Metrics

• Test files > 2,000 lines: 17 → 0
• Largest test file size: 4,511 lines → < 800 lines

---

Next Steps

1. Review and prioritise the tasks above
2. Assign tasks to Copilot coding agent via planner agent
3. Track progress on improvement items
4. Re-evaluate this focus area in 1 quarter (after splits land) or sooner if new files approach 2,000 lines

---

Generated by Repository Quality Improvement Agent
Next analysis: 2026-08-19 — Focus area selected by diversity algorithm

Generated by ⚡ Repository Quality Improvement Agent · auto · 31 AIC · ⌖ 8.49 AIC · ⊞ 9.9K · ◷

• expires on Aug 19, 2026, 5:14 AM UTC-08:00

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-08-19T13:14:08.108Z.

Closed by Workflow