Add comprehensive documentation for validation and parsing organization

[Copilot]

The semantic function clustering analysis identified validation functions, parse functions, and helper files as potential refactoring candidates. Analysis revealed these are already well-organized per documented architectural patterns—validation co-located with domain logic, parse functions grouped by purpose, helpers with 3+ callers.

Changes

Documentation Added

Added package-level documentation to 10 files explaining organization rationale:

Validation functions (7 functions across 5 files):

• sandbox.go - Sandbox-specific validation co-located with configuration logic
• repo_memory.go - Memory ID validation with memory implementation
• compile_campaign.go - Campaign validation with campaign compilation
• run_command.go - Input/remote workflow validation with run command
• actions_build_command.go - Action metadata validation with build logic

Parse functions (30+ functions across 2 files):

• tools_parser.go - All tool configuration parsing (github, bash, playwright, MCP, etc.)
• config_helpers.go - Safe output configuration parsing utilities

Helper files (11 files):

• engine_helpers.go - Shared agent/npm installation across engines
• map_helpers.go - Generic map/type conversion utilities
• compile_helpers.go - Batch compilation and statistics

Each doc explains: purpose, why functions are grouped here, usage patterns, and caller count justification.

Organization Principles

The codebase follows these patterns from pkg/workflow/validation.go and skills/developer/SKILL.md:

• Domain-specific validation in domain files - Not centralized unless cross-cutting
• Parse functions grouped by parsing domain - Tools vs configs vs time specs
• Helper files require 3+ callers and stable API - Not generic catch-alls

No code movement needed—current organization is sound.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• https://api.github.com/user
  • Triggering command: /usr/bin/gh gh api user --jq .login ithub/workflows stmain.go ache/go/1.25.0/x64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE ache/go/1.25.0/x64/pkg/tool/linux_amd64/vet env -json GO111MODULE 6520148/b372/vet.cfg GOINSECURE GOMOD GOMODCACHE /opt/hostedtoolcache/go/1.25.0/x/var/run/docker/runtime-runc/moby (http block)
  • Triggering command: /usr/bin/gh gh api user --jq .login g_.a om/org1/repo1 /opt/hostedtoolcache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env orkflows (http block)
  • Triggering command: /usr/bin/gh gh api user --jq .login ithub/workflows GO111MODULE 1f8519d77c713c6721a3659afe3dabfe56a/log.json l GOMOD GOMODCACHE /opt/hostedtoolcache/go/1.25.0/xremote.origin.url -ato�� -bool l (http block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Original prompt

---

This section details on the original issue you should resolve

<issue_title>[refactor] Semantic Function Clustering Analysis - Code Organization Improvements</issue_title>
<issue_description>## Executive Summary

This analysis examined 331 non-test Go source files across the repository to identify refactoring opportunities through semantic function clustering. The codebase demonstrates generally good organization with well-structured packages, but several opportunities for improvement were identified:

• 8 validation functions located outside dedicated validation files
• Multiple parse function clusters scattered across non-parser files (30+ functions in 10+ files)
• 11 helper files with overlapping responsibilities
• Similar render/generate function patterns across MCP configuration files (14 render functions)
• Opportunities to consolidate parsing logic for configuration extraction

The analysis focused on the largest packages: pkg/workflow (175 files) and pkg/cli (114 files), where the most significant refactoring opportunities exist.

Full Analysis Report

Repository Structure Overview

Package Distribution

• pkg/workflow: 175 files (52.9% of codebase) - Core workflow compilation and execution
• pkg/cli: 114 files (34.4%) - Command-line interface implementation
• pkg/parser: 20 files (6.0%) - Parsing and frontmatter extraction
• pkg/campaign: 8 files (2.4%) - Campaign orchestration
• Other packages: 14 files (4.3%) - Utilities, logging, console, styles

File Organization Assessment

The repository follows Go best practices with files organized by feature:

✅ Well-Organized Clusters:

• Create operations: 6 create_*.go files for entity creation (issues, PRs, discussions, etc.)
• Update operations: 7 update_*.go files for entity updates
• Compiler modules: 15 compiler_*.go files for workflow compilation stages
• Engine implementations: 5 *_engine.go files for different AI engines
• Safe output files: 13 safe_output*.go files for output validation
• MCP configuration: 24 mcp*.go files for MCP server integration
• Logs operations: 11 logs_*.go files in CLI for log analysis
• Update CLI commands: 9 update_*.go files in CLI for update operations

---

Identified Refactoring Opportunities

1. Validation Functions in Non-Validation Files

Issue: Validation functions scattered outside dedicated validation files

Outliers Found:

File: pkg/workflow/repo_memory.go

• Function: validateNoDuplicateMemoryIDs
• Issue: Memory validation function in memory implementation file
• Recommendation: Move to pkg/workflow/validation.go or create pkg/workflow/repo_memory_validation.go
• Impact: Improved separation of concerns

File: pkg/workflow/sandbox.go

• Functions: validateMountsSyntax, validateSandboxConfig
• Issue: Sandbox validation in main sandbox file
• Recommendation: Create pkg/workflow/sandbox_validation.go to match pattern of docker_validation.go, npm_validation.go, etc.
• Impact: Consistent validation file organization across features

Pattern Analysis: The codebase has 22 dedicated validation files following the *_validation.go naming pattern. These outliers break this established convention.

---

2. Parse Functions Scattered Across Files

Issue: 30+ parse functions distributed across 10+ non-parser files instead of being consolidated

Major Concentrations:

File: pkg/workflow/config_helpers.go (268 lines)
Parse functions:

• parseLabelsFromConfig
• parseTitlePrefixFromConfig
• parseTargetRepoFromConfig
• parseTargetRepoWithValidation
• parseParticipantsFromConfig
• parseAllowedReposFromConfig
• parseAllowedLabelsFromConfig
• parseExpiresFromConfig
• parseRelativeTimeSpec

Analysis: These are config parsing utilities, but the file is named "helpers" rather than "config_parser" which would be more accurate.

File: pkg/workflow/dependabot.go (685 lines)
Parse functions:

• parseNpmPackage
• parsePipPackage
• parseGoPackage

Issue: Package dependency parsing logic embedded in dependabot feature file
Recommendation: Extract to pkg/workflow/package_parsing.go for reusability

File: pkg/workflow/time_delta.go (370 lines)
Parse functions:

• parseTimeDelta
• parseTimeDeltaForStopAfter
• parseTimeDeltaWithMinutes
• parseAbsoluteDateTime
• parseRelativeDate

Analysis: This file is well-organized - all time parsing in one place ✅

Other Files with Parse Functions:

• pkg/workflow/map_helpers.go: parseIntValue
• pkg/workflow/reactions.go: parseReactionValue
• pkg/workflow/safe_inputs.go: parseSafeInputsMap
• pkg/workflow/safe_output_builder.go: parseRequiredLabelsFromConfig, parseRequiredTitlePrefixFromConfig
• pkg/workflow/safe_outputs_app.go: parseAppConfig
• `pkg/workflow/safe_outputs_config...

• Fixes [refactor] Semantic Function Clustering Analysis - Code Organization Improvements #7063

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.