C069: Plugin Extensibility — Validators & Custom Step Types

[pocky]

C069: Plugin Extensibility — Validators & Custom Step Types

Description

Extend the plugin protocol beyond operations to support two new capabilities: validators (custom workflow validation rules run during awf validate) and step_types (custom step execution logic for new type: values in workflow YAML). The plugin manifest already declares capabilities: [operations, commands, validators] but only operations is implemented. This chore replaces the unimplemented commands capability with step_types, adds domain ports for both new capabilities, extends the gRPC protocol, updates the SDK, and wires everything into WorkflowService and ExecutionService via the existing Set*() optional dependency pattern.

Tasks

• Define WorkflowValidatorProvider and StepTypeProvider ports in internal/domain/ports/plugin.go with ValidationResult, StepExecuteRequest, StepExecuteResult, and supporting types
• Add StepTypeChecker func(typeName string) bool parameter to Step.Validate() in internal/domain/workflow/step.go; accept unknown types when checker confirms, hard error when nil or returns false
• Replace CapabilityCommands with CapabilityStepTypes in internal/domain/pluginmodel/manifest.go ValidCapabilities
• Extend proto/plugin/v1/plugin.proto with ValidatorService (ValidateWorkflow, ValidateStep), StepTypeService (ListStepTypes, ExecuteStep), Severity enum (0=UNSPECIFIED treated as ERROR), and all associated message types
• Implement host-side validator gRPC client in internal/infrastructure/pluginmgr/grpc_validator.go with per-plugin timeout (default 5s), crash-as-timeout handling, result deduplication by (message + step + field)
• Implement host-side step type gRPC client in internal/infrastructure/pluginmgr/grpc_step_type.go with ListStepTypes() cache populated once at Init(), O(1) HasStepType() lookup, first-registered-wins conflict resolution with warning
• Update rpc_manager.go to delegate validator/step_type gRPC calls based on declared capability
• Add SetValidatorProvider() to WorkflowService; call validators after built-in validation in Validate(), merge and deduplicate results, display unified output
• Add SetStepTypeProvider() to ExecutionService; delegate unknown step types to plugin in executeStep(), convert StepExecuteResult to StepState, route exit codes through handleNonZeroExit
• Add --skip-plugins flag to awf validate and awf run; skip validators and step type lookup when set
• Add --validator-timeout flag to awf validate (default 5s)
• Extend SDK in pkg/plugin/sdk/ with Validator interface (ValidateWorkflow, ValidateStep), StepTypeProvider interface (StepTypes, ExecuteStep), WorkflowDefinition, StepDefinition, and helper types
• Create example validator plugin in examples/plugins/awf-plugin-security-validator/
• Create example step type plugin in examples/plugins/awf-plugin-database/
• Update docs/user-guide/plugins.md to document validator and step type capabilities
• Update docs/user-guide/workflow-syntax.md to document custom step types and config: field
• Update CHANGELOG.md to document commands → step_types capability change

Impact

• Files affected: internal/domain/ports/plugin.go, internal/domain/workflow/step.go, internal/domain/pluginmodel/manifest.go, proto/plugin/v1/plugin.proto, internal/infrastructure/pluginmgr/rpc_manager.go, internal/infrastructure/pluginmgr/grpc_validator.go (new), internal/infrastructure/pluginmgr/grpc_step_type.go (new), internal/application/workflow_service.go, internal/application/execution_service.go, pkg/plugin/sdk/sdk.go, pkg/plugin/sdk/validator.go (new), pkg/plugin/sdk/step_type.go (new), .go-arch-lint.yml, docs/user-guide/plugins.md, docs/user-guide/workflow-syntax.md, CHANGELOG.md
• Breaking changes: yes — commands capability removed from ValidCapabilities (never implemented, no known consumers); Step.Validate() signature gains StepTypeChecker parameter (all callers must update)
• Downtime required: no

Acceptance Criteria

• WorkflowValidatorProvider and StepTypeProvider ports defined in domain layer with zero infrastructure imports
• Step.Validate(nil) behaves identically to current behavior (backward compatible)
• Step.Validate(checker) accepts unknown types when checker returns true
• awf validate displays plugin validator results alongside built-in results with severity icons
• Validator timeout (default 5s) gracefully skips timed-out plugins with warning
• Validator crash treated as timeout, skipped with warning, other validators continue
• Multiple validator results deduplicated by (message + step + field)
• awf run delegates unknown step types to StepTypeProvider.ExecuteStep()
• Custom step output accessible via {{states.step_name.Output}} and {{states.step_name.Data.key}}
• Exit code from custom step flows through handleNonZeroExit for transition routing (F068)
• Retry works with custom step types (retry loop wraps gRPC ExecuteStep call)
• Pre/post hooks execute around custom step execution
• capture: on custom step type is silently ignored
• HasStepType() uses cached registration from ListStepTypes() at Init() — no per-check gRPC call
• Step type name conflict: first-registered wins, warning logged
• --skip-plugins skips validators in awf validate and step type lookup in awf run
• --skip-plugins with custom step type in workflow: awf run fails with clear explanation
• SetValidatorProvider(nil) and SetStepTypeProvider(nil) preserve existing behavior (no-op)
• SEVERITY_UNSPECIFIED (proto3 zero) treated as ERROR by host
• commands removed from ValidCapabilities, step_types added
• All existing tests pass without modification (except Step.Validate() callers updated)
• Unit tests cover all edge cases listed in testing strategy
• Integration tests verify end-to-end validator and step type plugin workflows
• go-arch-lint passes with any necessary mayDependOn updates
• Documentation updated for both capabilities

---

Metadata

• Status: backlog
• Version: v0.5.0
• Priority: medium
• Estimation: XL

Related

• Blocks: none
• Blocked by: F077 (gRPC transport), F078 (plugin registry/distribution)
• Related issues: F066 (inline on_failure), F068 (exit code routing), C062 (agent state options audit)

Notes

Capability commands removal: commands was declared in ValidCapabilities but never implemented. No external plugins exist yet (plugin system non-functional until F077). This is a clean removal, not a deprecation requiring migration.

Step.Validate() signature change: Adding StepTypeChecker parameter is a compile-time breaking change for all callers. The func(string) bool pattern mirrors the existing ExpressionCompiler func(string) error precedent in Workflow.Validate(). All callers passing nil get identical behavior to today.

capture: limitation: Custom step types return structured output + data, not stdout/stderr streams. capture: is designed for stream splitting and is incompatible with the gRPC response model. Document as v1 limitation; users access output via state interpolation.

Retry integration: The existing retry loop in executeStepCommand wraps the execution call. Custom step types slot into the same position — the gRPC ExecuteStep call replaces the shell command call inside the loop. No retry infrastructure changes needed.

Step type registration caching: ListStepTypes() is called once after Init() succeeds. The cache lives on the pluginConnection struct. Cache invalidation occurs only on plugin shutdown/restart. This avoids N gRPC round-trips per workflow validation or execution.

Validator JSON serialization: Validators receive workflow_json (JSON-encoded domain struct), not raw YAML. This avoids filesystem re-reads, ensures consistent representation, and lets plugin authors work with a structured schema. The SDK provides WorkflowDefinition for convenient unmarshaling.