Conversation
…ecking Adds three new wfctl commands to ensure templates remain valid across engine releases and that application configs don't introduce breaking changes: - `wfctl template validate`: validates project templates and config files against the engine's known module/step types, dependency resolution, trigger types, and config field warnings (strict mode available) - `wfctl contract test`: generates API contract snapshots from configs (endpoints, modules, steps, events) and compares against baselines to detect breaking changes (removed endpoints, added auth requirements) - `wfctl compat check`: verifies all module and step types in a config are available in the current engine version Also adds `cmd/wfctl/type_registry.go` with a static registry of all ~50 module types and ~40 step types extracted from plugin packages. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This pull request adds comprehensive validation, contract testing, and compatibility checking capabilities to the wfctl CLI tool. The PR introduces three new command groups that enable developers to validate workflow configurations against the engine's known types, generate and compare API contracts to detect breaking changes, and verify compatibility between configs and engine versions.
Changes:
- Adds
wfctl template validatecommand to validate project templates and workflow configs against known module/step types with optional strict mode - Adds
wfctl contract testcommand to generate API contract snapshots (endpoints, modules, steps, events) and compare against baselines to detect breaking changes - Adds
wfctl compat checkcommand to verify all module/step types in a config are available in the current engine version
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| cmd/wfctl/main.go | Registers three new command handlers: template, contract, and compat |
| cmd/wfctl/type_registry.go | Defines static registry of ~50 module types and ~40 step types with metadata (plugin, stateful flag, config keys) |
| cmd/wfctl/type_registry_test.go | Comprehensive tests for type registry validation (coverage for populated types, plugin fields, stateful flags, naming conventions) |
| cmd/wfctl/template_validate.go | Implements template validation logic: renders templates with placeholder values, validates module/step types, checks dependencies, reports warnings on unknown config fields |
| cmd/wfctl/template_validate_test.go | Tests for template validation across all templates, config files, error cases, strict mode, and edge cases |
| cmd/wfctl/contract.go | Generates contracts from configs (endpoints, modules, events), compares contracts to detect breaking changes (removed endpoints, auth added to public routes) |
| cmd/wfctl/contract_test.go | Tests for contract generation, comparison logic, breaking change detection, and JSON output |
| cmd/wfctl/compat.go | Checks config compatibility by verifying module/step types against known registry, includes custom insertion sort implementation |
| cmd/wfctl/compat_test.go | Tests for compatibility checking with valid/invalid configs, JSON output, and edge cases |
| for key := range stepCfg { | ||
| if !knownKeys[key] { | ||
| result.Warnings = append(result.Warnings, fmt.Sprintf("pipeline %q step %q (%s) config field %q not in known fields", pipelineName, stepMap["name"], stepType, key)) |
There was a problem hiding this comment.
The warning message on line 412 accesses stepMap["name"] without checking if it exists or is a string. If a step doesn't have a "name" field, this will format as <nil> in the output string, which is not user-friendly.
Consider adding a type assertion:
stepName, _ := stepMap["name"].(string)
if stepName == "" {
stepName = "(unnamed)"
}
result.Warnings = append(result.Warnings, fmt.Sprintf("pipeline %q step %q (%s) config field %q not in known fields", pipelineName, stepName, stepType, key))| for key := range stepCfg { | |
| if !knownKeys[key] { | |
| result.Warnings = append(result.Warnings, fmt.Sprintf("pipeline %q step %q (%s) config field %q not in known fields", pipelineName, stepMap["name"], stepType, key)) | |
| stepName, _ := stepMap["name"].(string) | |
| if stepName == "" { | |
| stepName = "(unnamed)" | |
| } | |
| for key := range stepCfg { | |
| if !knownKeys[key] { | |
| result.Warnings = append(result.Warnings, fmt.Sprintf("pipeline %q step %q (%s) config field %q not in known fields", pipelineName, stepName, stepType, key)) |
| // sortCompatItems sorts compat items by type name. | ||
| func sortCompatItems(items []compatItem) { | ||
| for i := 1; i < len(items); i++ { | ||
| for j := i; j > 0 && items[j].Type < items[j-1].Type; j-- { | ||
| items[j], items[j-1] = items[j-1], items[j] | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
This manual insertion sort implementation should be replaced with the standard library's sort.Slice for consistency with the rest of the codebase. In contract.go, sort.Slice is used throughout (lines 232, 321, 335, 414, 454, 496), and using the same pattern here would improve maintainability and follow established conventions.
Replace this function with:
sort.Slice(items, func(i, j int) bool {
return items[i].Type < items[j].Type
})| // templateFSReader allows reading from the embedded templateFS for validation. | ||
| // It wraps around the existing templateFS embed.FS. | ||
| var _ fs.FS = templateFS |
There was a problem hiding this comment.
The comment on line 530 describes a wrapper around templateFS, but line 531 only contains a compile-time interface check that doesn't actually wrap or extend any functionality. Either remove the misleading comment or implement an actual wrapper if one is needed.
The compile-time check var _ fs.FS = templateFS is valid but the comment is inaccurate since templateFS is already defined elsewhere (in init.go line 15) and this is just verifying it implements fs.FS.
| // templateFSReader allows reading from the embedded templateFS for validation. | |
| // It wraps around the existing templateFS embed.FS. | |
| var _ fs.FS = templateFS | |
| // Compile-time check that templateFS implements fs.FS for use in validation helpers. | |
| var _ fs.FS = templateFS |
Summary
wfctl template validateto validate project templates and workflow config files against the engine's known module/step types, dependency resolution, trigger types, and unknown config field detection (with--strictmode)wfctl contract testto generate API contract snapshots (endpoints, modules, steps, events) from configs and compare against baselines to detect breaking changes (removed endpoints, auth added to public routes)wfctl compat checkto verify all module/step types in a config are available in the current engine versioncmd/wfctl/type_registry.gowith a complete static registry of all ~50 module types and ~40 step types extracted from all plugin packagesTest plan
go build ./cmd/wfctl/succeedsgo test ./cmd/wfctl/...— all tests pass (9 new test files, ~60 new test cases)wfctl template validate— validates all 5 project templates (api-service, event-processor, full-stack, plugin, ui-plugin)wfctl contract test config.yaml— generates contract JSONwfctl contract test -baseline baseline.json -output current.json config.yaml— detects breaking changeswfctl compat check config.yaml— reports module/step type availability🤖 Generated with Claude Code