Problem
When running a workflow normally, validation is weaker than when using --validate-only. This causes:
- Unknown node types to show ugly tracebacks instead of clean error messages
- Validation errors to be caught at compilation time instead of validation time
- Inconsistent user experience between validation modes
Steps to Reproduce
Create a workflow with an unknown node type:
{
"ir_version": "0.1.0",
"nodes": [
{"id": "step1", "type": "shell", "params": {"command": "echo hello"}},
{"id": "step2", "type": "nonexistent-node", "params": {}}
],
"edges": [{"from": "step1", "to": "step2"}]
}
With --validate-only (clean error):
$ pflow --validate-only workflow.json
Validating workflow (static validation)...
✗ Static validation failed:
• Unknown node type: 'nonexistent-node'
Normal execution (ugly traceback):
$ pflow workflow.json
ERROR: Node instantiation failed
Traceback (most recent call last):
...
pflow.runtime.compiler.CompilationError: compiler: Node type 'nonexistent-node' not found
...
❌ Planning failed: compiler: Node type 'nonexistent-node' not found
Root Cause
The CLI has two different validation paths:
| Mode |
Validation Used |
--validate-only |
WorkflowValidator.validate() - full 5-layer validation |
| Normal execution |
validate_ir() - schema-only validation |
The normal execution path uses _validate_and_handle_workflow_errors() which only calls validate_ir() for schema validation. It does NOT validate:
- Node types exist in registry
- Data flow correctness (cycles, forward refs)
- Template variable resolution
- Output source validity
Expected Behavior
- Normal execution should use the same validation as
--validate-only
- Unknown node types should be caught with clean error messages
- Validation should happen BEFORE any nodes execute
- Error messages should be consistent across all modes
Proposed Solution
- Replace schema-only validation with full
WorkflowValidator.validate()
- Use real execution params (not dummy) for template validation
- Remove dead code (
_validate_workflow_structure, _validate_and_handle_workflow_errors)
- Add tests verifying validation happens before execution
Impact
- User Experience: Clean, consistent error messages
- Safety: Validation catches more issues before execution starts
- Code Quality: Removes duplicate validation logic (~65 lines)
Problem
When running a workflow normally, validation is weaker than when using
--validate-only. This causes:Steps to Reproduce
Create a workflow with an unknown node type:
{ "ir_version": "0.1.0", "nodes": [ {"id": "step1", "type": "shell", "params": {"command": "echo hello"}}, {"id": "step2", "type": "nonexistent-node", "params": {}} ], "edges": [{"from": "step1", "to": "step2"}] }With
--validate-only(clean error):$ pflow --validate-only workflow.json Validating workflow (static validation)... ✗ Static validation failed: • Unknown node type: 'nonexistent-node'Normal execution (ugly traceback):
Root Cause
The CLI has two different validation paths:
--validate-onlyWorkflowValidator.validate()- full 5-layer validationvalidate_ir()- schema-only validationThe normal execution path uses
_validate_and_handle_workflow_errors()which only callsvalidate_ir()for schema validation. It does NOT validate:Expected Behavior
--validate-onlyProposed Solution
WorkflowValidator.validate()_validate_workflow_structure,_validate_and_handle_workflow_errors)Impact