Implement golden file testing for compiler output validation

[Copilot]

Adds golden file testing pattern to validate compiler-generated YAML and catch regressions. Previously, compiler output was validated through manual inspection or brittle string matching.

Implementation

• Testing infrastructure (testing_helpers.go)

  • CompareGoldenFile() helper with -update-golden flag support
  • Automatic golden file creation and comparison

• Test suite (compiler_golden_test.go)

  • Table-driven tests covering 10 workflow patterns
  • Automatic temp directory setup/teardown
  • Import file handling for shared configuration tests

• Test coverage (testdata/{workflows,golden})

  • Issue/PR/schedule/command triggers
  • Multi-job workflows
  • MCP servers, safe outputs, network permissions
  • Imports and custom engines

• Makefile target (make update-golden)

  • Convenient target for updating golden files
  • Provides clear feedback and instructions

• Documentation (specs/golden-file-testing.md)

  • Comprehensive specification in specs folder
  • Architecture, implementation, and usage details
  • Quick reference in TESTING.md

Usage

# Normal test run - fails if output differs
go test ./pkg/workflow -run TestCompilerGoldenFiles

# Update golden files after intentional compiler changes
make update-golden

# Or use the test flag directly
go test ./pkg/workflow -run TestCompilerGoldenFiles -update-golden

Tests execute in ~130ms total. Full documentation available in specs/golden-file-testing.md.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>[task] Implement golden file testing for compiler output validation</issue_title>
<issue_description>## Objective
Implement golden file testing pattern to capture expected compiler output and detect unintended changes in generated workflows.

Context

The workflow compiler generates complex YAML output currently validated through manual inspection or brittle string matching. Golden file testing will improve test maintainability and catch regressions. Part of discussion #3770.

Approach

1. Create testdata structure

pkg/workflow/testdata/
├── workflows/
│   ├── issue_trigger.md
│   ├── pr_trigger.md
│   ├── scheduled.md
│   └── ...
└── golden/
    ├── issue_trigger.lock.yml
    ├── pr_trigger.lock.yml
    ├── scheduled.lock.yml
    └── ...

2. Create golden file helper

Create pkg/workflow/testing_helpers.go with:

func CompareGoldenFile(t *testing.T, got []byte, goldenPath string, update bool) {
    // Implementation to compare output with golden file
    // Support --update flag to regenerate golden files
}

3. Update existing compiler tests

• Replace string-based output validation with golden file comparisons
• Add table-driven tests for multiple workflow patterns
• Test various trigger types, permissions, and tool configurations

4. Add test flag support

• Support -update-golden test flag to regenerate golden files
• Document usage in TESTING.md

Representative Golden Files (minimum 10)

• Issue trigger workflows
• Pull request trigger workflows
• Scheduled workflows
• Command trigger workflows
• Multi-job workflows
• Workflows with MCP servers
• Workflows with safe outputs
• Workflows with network permissions
• Workflows with imports
• Workflows with custom engines

Files to Create/Modify

• Create: pkg/workflow/testdata/ directory structure
• Create: pkg/workflow/testing_helpers.go
• Modify: pkg/workflow/compiler_test.go
• Update: TESTING.md

Acceptance Criteria

• testdata directory structure created
• Golden file comparison helper implemented
• At least 10 representative workflow examples with golden files
• Existing compiler tests updated to use golden files
• -update-golden flag support added
• Documentation added to TESTING.md
• All tests pass with make test-unit
  Related to 🎯 Repository Quality Improvement Report - Testing Infrastructure #3770

AI generated by Plan Command for discussion #3770</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes [task] Implement golden file testing for compiler output validation #3773

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[pelikhan]

@copilot add makefile target to update golden file, document process

[pelikhan]

@copilot move file to specs folder

[Copilot]

Moved golden file testing documentation to specs/golden-file-testing.md and updated TESTING.md to reference it (commit 8fdd9b8).

The specs file includes complete documentation on architecture, implementation, usage, and benefits. TESTING.md now has a streamlined quick reference section that links to the detailed spec.

[Copilot]

@copilot add makefile target to update golden file, document process

Added make update-golden Makefile target and documented the process in specs/golden-file-testing.md (commit 8fdd9b8).

Usage:

make update-golden

The target runs the golden file tests with the -update-golden flag and provides feedback about reviewing changes.