[repository-quality] 🎯 Repository Quality Improvement Report - Large File Decomposition in pkg/workflow

[github-actions[bot]]

🎯 Repository Quality Improvement Report - Large File Decomposition in pkg/workflow

Analysis Date: 2026-08-17
Focus Area: Large File Decomposition (Code Organization, custom variant)
Strategy Type: Custom
Custom Area: Yes — pkg/workflow is the compiler's core package (479 non-test Go files, 275,630 source LOC) and is the most actively evolved part of gh-aw. The repo's own developer-code-organization skill emphasizes splitting large files by responsibility, but no automated check enforces this, so several files have grown past a size where they're easy to review or safely modify. Targeting this now reduces future review friction and merge-conflict risk as the compiler keeps growing.

Executive Summary

pkg/workflow contains 14 files over 800 lines, several combining multiple responsibilities (parsing, validation, string/YAML generation, and compiler orchestration) in one file. The three largest — compiler_custom_jobs.go (1142 lines, 37 funcs), cache.go (1118 lines, 36 funcs), and dependabot.go (1072 lines, 33 funcs) — each mix distinct sub-domains: for example, cache.go combines cache-memory config parsing, generic cache step YAML generation, and cache-memory step/job generation; dependabot.go combines npm/pip/go manifest generation with dependabot.yml reconciliation logic.

Splitting these files along their existing functional seams (which are already visible from function naming, e.g. parseCacheMemory* vs writeCacheStep* vs generateCacheMemory*) would let contributors and Copilot review/modify one concern without wading through unrelated code, and reduces the blast radius of future changes. This is a mechanical, low-risk refactor: pure code movement with no behavior change, verified by make test-unit and make build.

We recommend prioritizing cache.go and dependabot.go first since their internal seams are the cleanest (parsing vs YAML-writing vs job-building), followed by compiler_custom_jobs.go, and finally addressing the other 11 files >800 lines as a follow-up sweep.

Full Analysis Report

Focus Area: Large File Decomposition in pkg/workflow

Current State Assessment

pkg/workflow is the largest and most frequently modified package in gh-aw, holding the markdown-to-YAML compiler core. As the compiler has grown feature-by-feature (custom jobs, cache memory, dependabot manifests, safe-outputs handlers, triggers, etc.), several files have accreted multiple unrelated responsibilities rather than being split when they crossed common size thresholds (500–800 lines).

Metrics Collected:

Metric	Value	Status
Non-test .go files in pkg/workflow	479	✅
Files >800 lines	14	⚠️
Largest file	compiler_custom_jobs.go (1142 lines, 37 funcs)	⚠️
Test LOC vs Source LOC (repo-wide)	537,691 vs 275,630 (1.95x)	✅ Healthy
Avg lines/file in pkg/workflow	260	✅
TODO/FIXME count (repo-wide)	37	✅

Findings

Strengths

• Test coverage ratio is strong (~2x test LOC vs source LOC), so refactors here can be validated with confidence.
• Average file size in pkg/workflow (260 lines) is healthy; the problem is concentrated in a small number of outlier files (14 of 479, ~3%).
• Existing function naming within the large files (parseCacheMemory*, writeCacheStep*, generateCacheMemory*, generate*Manifests, reconcile*) already reveals natural split boundaries — this is a low-risk, mechanical refactor.

Areas for Improvement

• ⚠️ Medium: pkg/workflow/cache.go (1118 lines, 36 funcs) mixes three concerns: cache-memory config parsing (parseCacheMemory*, lines 134–297), generic cache: step YAML generation (generateCacheSteps, writeCacheStep*, lines 384–521), and cache-memory-specific step/job generation (generateCacheMemory*, buildUpdateCacheMemoryJob, lines 530–1010+).
• ⚠️ Medium: pkg/workflow/dependabot.go (1072 lines, 33 funcs) mixes ecosystem-specific manifest generation (npm/pip/go, lines 137–1044) with dependabot.yml reconciliation logic (ReconcileManagedDependabotIgnores*, lines 493–719) — two largely independent concerns sharing one file.
• ⚠️ Medium: pkg/workflow/compiler_custom_jobs.go (1142 lines, 37 funcs) mixes custom-job property extraction (extractCustomJob*, lines 173–433), built-in job augmentation (applyBuiltinJob*, extractBuiltinJob*, lines 598–984), and pinned-step/checkout helpers (lines 1068–1142).
• i️ Low: 11 additional files (safe_outputs_handler_registry.go, compiler_safe_outputs_job.go, workflow_builder.go, trigger_parser.go, awf_config.go, domains.go, behavior_defined_engine.go, compiler_pre_activation_job.go, observability_otlp.go, tools_parser.go, markdown_security_scanner.go) also exceed 800 lines but weren't deep-dived in this pass; worth a follow-up sweep.

Detailed Analysis

No automated lint currently flags file size or single-responsibility violations in this codebase (only go vet/custom linters per go-linters skill are wired up). Because pkg/workflow changes weekly (compiler is under active development), oversized files compound: every new feature has an incentive to append to the nearest existing file rather than create a new one, since the existing large files already "own" the relevant struct/helper. Splitting the three worst offenders now, along their pre-existing functional seams, arrests this trend before the files grow further and makes room for a lightweight follow-up convention (e.g., a soft file-size guideline in developer-code-organization skill or a CI warning).

---

🤖 Tasks for Copilot Agent

NOTE TO PLANNER AGENT: Split the following tasks into individual work items.

Improvement Tasks

Task 1: Split pkg/workflow/cache.go into config-parsing, generic-cache, and cache-memory files

Priority: High
Estimated Effort: Medium
Focus Area: Code Organization

Description: cache.go (1118 lines) mixes cache-memory config parsing, generic cache: step generation, and cache-memory job/step generation. Split into three files: cache_memory_config.go (config parsing, lines ~42–382), cache_steps.go (generic cache step YAML generation, lines ~384–530), and cache_memory_steps.go (cache-memory-specific step/job generation, lines ~530 end). Keep package workflow; move code verbatim with git mv-style extraction (no logic changes).

Acceptance Criteria:

• cache.go split into 3 new files with no behavior change
• All existing tests in pkg/workflow/cache_test.go (and related) pass unchanged
• make build and make test-unit succeed
• No file in the split exceeds ~600 lines

Code Region: pkg/workflow/cache.go

Split the file `pkg/workflow/cache.go` in the gh-aw Go repository into three files without changing any behavior:

1. `pkg/workflow/cache_memory_config.go` — cache-memory config parsing: `CacheMemoryConfig`, `CacheMemoryEntry` types, `cacheMemoryDirFor`, `cacheMemoryValidationStepID`, `cacheHasValidationStep`, `isValidCacheID`, `isValidFileExtension`, `generateDefaultCacheKey`, `parseCacheMemoryEntry`, `parseCacheMemoryIdentity`, `ensureCacheRunIDSuffix`, `parseCacheMemoryDescription`, `parseCacheMemoryRetentionDays`, `parseOptionalInt`, `parseCacheMemoryRestoreOnly`, `parseCacheMemoryScope`, `parseCacheMemoryAllowedExtensions`, `applyDefaultAllowedExtensions`, `(c *Compiler) extractCacheMemoryConfig`, `defaultCacheMemoryEntries`, `parseCacheMemoryEntries`, `(c *Compiler) extractCacheMemoryConfigFromMap`.
2. `pkg/workflow/cache_steps.go` — generic `cache:` step YAML generation: `generateCacheSteps`, `parseCacheStepConfigs`, `normalizeCacheStepArray`, `writeCacheStep`, `resolveCacheStepName`, `writeCachePath`, `buildCacheRestoreKeys`, `writeCacheRestoreKeys`, `writeCacheStepValue`.
3. `pkg/workflow/cache_memory_steps.go` — cache-memory-specific step/job generation: `generateCacheMemorySteps`, `generateCacheMemoryGitSetupStep`, `generateCacheMemoryGitCommitSteps`, `generateCacheMemoryValidation`, `generateCacheMemoryArtifactUpload`, `buildCacheMemoryPromptSection`, `(c *Compiler) buildUpdateCacheMemoryJob`.

Keep the package declaration `package workflow` and all necessary imports in each new file (only import what's used per file). Delete `cache.go` once its contents are fully moved, or leave a minimal remaining file if any small helper doesn't fit the three groups cleanly — in that case put it in whichever new file makes the most sense. Do not change any function signatures, logic, or comments beyond moving them. After splitting, run `make fmt`, `make build`, and `make test-unit` (targeted at `pkg/workflow`) to confirm no regressions, and ensure no single new file exceeds ~600 lines.

---

Task 2: Split pkg/workflow/dependabot.go into manifest-generation and ignore-reconciliation files

Priority: High
Estimated Effort: Medium
Focus Area: Code Organization

Description: dependabot.go (1072 lines) mixes npm/pip/go manifest generation with dependabot.yml "ignore" reconciliation logic. Split into dependabot_manifests.go (ecosystem manifest generation) and dependabot_ignores.go (reconciliation of managed ignore entries), keeping shared types (DependabotConfig, DependabotUpdateEntry) in whichever file is imported by both, or a small shared dependabot_types.go.

Acceptance Criteria:

• dependabot.go split into 2–3 new files with no behavior change
• All existing tests (pkg/workflow/dependabot_test.go etc.) pass unchanged
• make build and make test-unit succeed
• No new file exceeds ~600 lines

Code Region: pkg/workflow/dependabot.go

Split the file `pkg/workflow/dependabot.go` in the gh-aw Go repository into files without changing any behavior:

1. `pkg/workflow/dependabot_types.go` — shared types: `PackageJSON`, `DependabotConfig`, `DependabotUpdateEntry`, `NpmDependency`, `PipDependency`, `GoDependency`.
2. `pkg/workflow/dependabot_manifests.go` — manifest generation: `(c *Compiler) GenerateDependabotManifests`, `(c *Compiler) handleManifestGenerationError`, `(c *Compiler) generateNpmManifests`, `(c *Compiler) generatePipManifests`, `(c *Compiler) generateGoManifests`, `(c *Compiler) collectNpmDependencies`, `parseNpmPackage`, `(c *Compiler) loadOrInitPackageJSON`, `(c *Compiler) generatePackageJSON`, `(c *Compiler) generatePackageLock`, `(c *Compiler) collectPipDependencies`, `parsePipPackage`, `(c *Compiler) mergeExistingRequirements`, `(c *Compiler) generateRequirementsTxt`, `(c *Compiler) collectGoDependencies`, `parseGoPackage`, `extractGoPackages`, `extractGoFromCommands`, `(c *Compiler) loadOrInitGoModLines`, `appendGoModRequireSection`, `(c *Compiler) generateGoMod`.
3. `pkg/workflow/dependabot_ignores.go` — `dependabot.yml` reconciliation: `loadOrInitDependabotConfig`, `addMissingDependabotEcosystems`, `(c *Compiler) generateDependabotConfig`, `(c *Compiler) ReconcileManagedDependabotIgnores`, `reconcileGithubActionsIgnoreResult`, `reconcileGithubActionsIgnoreEntry`, `DependabotConfigPath`, `(c *Compiler) ReconcileManagedDependabotIgnoresInRepo`, `dependabotToAnySlice`, `dependabotToStringAnyMap`, `isYAMLNullOrEmptyScalar`, `managedPatternsWithInlineComment`, `normalizeDependabotIgnoreEntries`.

Keep `package workflow` and correct imports in each file. Move code verbatim with no logic changes. Delete `dependabot.go` once fully split. Run `make fmt`, `make build`, and `make test-unit` afterward to confirm all `pkg/workflow` tests (especially dependabot-related ones) still pass, and ensure no new file exceeds ~600 lines.

---

Task 3: Split pkg/workflow/compiler_custom_jobs.go by extraction / augmentation / pinned-steps concerns

Priority: Medium
Estimated Effort: Medium
Focus Area: Code Organization

Description: compiler_custom_jobs.go (1142 lines) mixes custom-job property extraction, built-in job augmentation, and pinned-step/checkout helpers. Split into compiler_custom_job_extraction.go, compiler_builtin_job_augmentation.go, and compiler_pinned_steps.go.

Acceptance Criteria:

• File split into 3 new files with no behavior change
• All existing tests pass unchanged
• make build and make test-unit succeed
• No new file exceeds ~600 lines

Code Region: pkg/workflow/compiler_custom_jobs.go

Split the file `pkg/workflow/compiler_custom_jobs.go` in the gh-aw Go repository into three files without changing any behavior:

1. `pkg/workflow/compiler_custom_job_extraction.go` — custom job building and property extraction: `(c *Compiler) buildCustomJobs`, `(c *Compiler) getCustomJobDependencySets`, `(c *Compiler) shouldSkipCustomJob`, `(c *Compiler) buildCustomJob`, `extractCustomJobNeeds`, `(c *Compiler) applyAutomaticActivationDependency`, `(c *Compiler) extractCustomJobProperties`, `(c *Compiler) extractCustomJobCoreProperties`, `(c *Compiler) extractCustomJobRunsOn`, `extractCustomJobTimeoutMinutes`, `extractCustomJobConcurrency`, `extractCustomJobEnv`, `extractCustomJobContainer`, `extractCustomJobServices`, `extractCustomJobContinueOnError`, `extractCustomJobEnvironment`, `extractCustomJobOutputs`, `(c *Compiler) configureCustomJobExecution`, `configureCustomReusableWorkflow`, `(c *Compiler) configureCustomJobSteps`, `formatIndentedYAMLField`.
2. `pkg/workflow/compiler_builtin_job_augmentation.go` — built-in job pre-steps and augmentation: `(c *Compiler) applyBuiltinJobPreSteps`, `insertActivationStepsBeforeArtifactStaging`, `normalizeBuiltinJobAlias`, `extractBuiltinJobNeedsAugmentation`, `extractBuiltinJobIfAugmentation`, `(c *Compiler) applyBuiltinJobAugmentations`, `applyBuiltinJobPermissionsAugmentation`, `(c *Compiler) guardIfAgainstStatusFuncBypass`, `ifExpressionContainsStatusFunc`, `validateRestrictedBuiltinSetupSteps`, `validateRestrictedBuiltinSteps`, `insertSetupStepsAtStart`, `insertPreStepsAtEarliestBoundary`.
3. `pkg/workflow/compiler_pinned_steps.go` — pinned job steps and checkout helpers: `(c *Compiler) extractPinnedJobSteps`, `ensureCheckoutPersistCredentials`, `isCheckoutAction`.

Keep `package workflow` and correct imports in each file. Move code verbatim, no logic changes. Delete `compiler_custom_jobs.go` once fully split. Run `make fmt`, `make build`, and `make test-unit` afterward to confirm all `pkg/workflow` tests still pass, and ensure no new file exceeds ~600 lines.

---

Task 4: Audit and split remaining 11 files over 800 lines in pkg/workflow

Priority: Medium
Estimated Effort: Large
Focus Area: Code Organization

Description: Beyond the top 3 files, 11 more files exceed 800 lines: safe_outputs_handler_registry.go (1082), compiler_safe_outputs_job.go (1065), workflow_builder.go (1059), trigger_parser.go (1023), awf_config.go (1017), domains.go (1014), behavior_defined_engine.go (1002), compiler_pre_activation_job.go (962), observability_otlp.go (942), tools_parser.go (818), markdown_security_scanner.go (804). Each needs its own responsibility audit before splitting.

Acceptance Criteria:

• Each of the 11 files reviewed for split opportunities along functional seams
• Files with clear multi-responsibility mixing (similar to Tasks 1–3) are split; files that are cohesive single-purpose files (e.g., long switch/case tables) are documented as intentionally large
• make build and make test-unit succeed after any splits

Code Region: pkg/workflow/{safe_outputs_handler_registry,compiler_safe_outputs_job,workflow_builder,trigger_parser,awf_config,domains,behavior_defined_engine,compiler_pre_activation_job,observability_otlp,tools_parser,markdown_security_scanner}.go

In the gh-aw Go repository, audit these 11 files in `pkg/workflow`, each currently over 800 lines: `safe_outputs_handler_registry.go`, `compiler_safe_outputs_job.go`, `workflow_builder.go`, `trigger_parser.go`, `awf_config.go`, `domains.go`, `behavior_defined_engine.go`, `compiler_pre_activation_job.go`, `observability_otlp.go`, `tools_parser.go`, `markdown_security_scanner.go`.

For each file:
1. List its top-level functions/types (`grep -n '^func \|^type '`) and group them by responsibility based on naming and behavior.
2. If the file mixes 2+ clearly separable responsibilities (like parsing vs. YAML generation vs. job building), split it into multiple files following the `<original_name>_<responsibility>.go` naming convention, moving code verbatim with no logic changes.
3. If the file is already a single cohesive responsibility (e.g., a long lookup table, a single large switch/case dispatcher, or a naturally sequential algorithm), leave it as-is and add a one-line comment at the top noting it was reviewed and intentionally kept together.

Keep `package workflow` and correct imports in every new file. Do not change any function signatures, logic, or exported behavior. After all splits, run `make fmt`, `make build`, and `make test-unit` to confirm no regressions, and ensure no new file exceeds ~600 lines unless explicitly justified as cohesive.

---

📊 Historical Context

Previous Focus Areas

Date	Focus Area	Type	Custom	Key Outcomes
2026-08-17	Large File Decomposition in pkg/workflow	Custom	Y	First run — established baseline: 14 files >800 lines in pkg/workflow

---

🎯 Recommendations

Immediate Actions (This Week)

1. Split pkg/workflow/cache.go into 3 responsibility-focused files — Priority: High
2. Split pkg/workflow/dependabot.go into manifest vs. ignore-reconciliation files — Priority: High

Short-term Actions (This Month)

1. Split pkg/workflow/compiler_custom_jobs.go along extraction/augmentation/pinned-steps seams — Priority: Medium
2. Audit and split the remaining 11 files >800 lines — Priority: Medium

Long-term Actions (This Quarter)

1. Add a soft file-size guideline (e.g., ~600 lines) to the developer-code-organization skill and consider a CI advisory check — Priority: Low

---

📈 Success Metrics

• Files >800 lines in pkg/workflow: 14 → 0 (or explicitly justified exceptions)
• Largest file in pkg/workflow: 1142 lines → ≤600 lines
• Avg lines/file in pkg/workflow: 260 → maintained or reduced despite ongoing feature growth

---

Next Steps

1. Review and prioritise the tasks above
2. Assign tasks to Copilot coding agent via planner agent
3. Track progress on improvement items
4. Re-evaluate this focus area in 1 quarter

---

Generated by Repository Quality Improvement Agent
Next analysis: 2026-08-18 — Focus area selected by diversity algorithm

Generated by ⚡ Repository Quality Improvement Agent · auto · 27.5 AIC · ⌖ 2.2 AIC · ⊞ 9.7K · ◷

• expires on Aug 18, 2026, 5:12 AM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Repository Quality Improvement Agent.

A newer discussion is available at Discussion #53697.