Summary
When an LLM node fails in a pflow workflow, the error message displayed to users omits the actual exception details, making debugging impossible. Additionally, the workflow trace file incorrectly reports success: true for failed nodes and final_status: "success" for failed workflows.
Issues
Issue 1: LLM Error Messages Swallow Exception Details
Severity: High - Blocks debugging
When an LLM call fails for reasons other than UnknownModelError or NeedsKeyException, the original exception message is discarded and replaced with a generic message.
Steps to Reproduce:
cat > /tmp/test-llm-fail.json << 'EOF2'
{
"inputs": {},
"nodes": [{"id": "test-llm", "type": "llm", "params": {"prompt": "Say hello", "model": "nonexistent-model"}}],
"edges": []
}
EOF2
pflow /tmp/test-llm-fail.json
Actual behavior:
Message: LLM call failed after 3 attempts. Model: nonexistent-model
Expected behavior:
Message: LLM call failed after 3 attempts. Model: nonexistent-model. Error: UnknownModelError...
Root Cause: In src/pflow/nodes/llm/llm.py, the exec_fallback method captures error_msg = str(exc) but doesn't include it in the generic error case.
Issue 2: Trace File Reports Incorrect Success Status
Severity: Medium - Misleading debug information
The workflow trace file reports success: true even when the workflow fails via error action.
Actual trace output:
{
"final_status": "success",
"nodes": [{"node_id": "test-llm", "success": true}]
}
Expected trace output:
{
"final_status": "failed",
"nodes": [{"node_id": "test-llm", "success": false}]
}
Root Cause: In src/pflow/runtime/instrumented_wrapper.py, trace is recorded with success=True before checking if the node returned "error" action. The caching logic already checks result != "error" but trace recording doesn't.
Impact
- AI agents cannot debug workflow failures - No way to determine why LLM calls fail
- Trace files are unreliable - Cannot be trusted to indicate actual success/failure status
Environment
- pflow version: current main
- Platform: macOS/Linux
Summary
When an LLM node fails in a pflow workflow, the error message displayed to users omits the actual exception details, making debugging impossible. Additionally, the workflow trace file incorrectly reports
success: truefor failed nodes andfinal_status: "success"for failed workflows.Issues
Issue 1: LLM Error Messages Swallow Exception Details
Severity: High - Blocks debugging
When an LLM call fails for reasons other than
UnknownModelErrororNeedsKeyException, the original exception message is discarded and replaced with a generic message.Steps to Reproduce:
Actual behavior:
Expected behavior:
Root Cause: In
src/pflow/nodes/llm/llm.py, theexec_fallbackmethod captureserror_msg = str(exc)but doesn't include it in the generic error case.Issue 2: Trace File Reports Incorrect Success Status
Severity: Medium - Misleading debug information
The workflow trace file reports
success: trueeven when the workflow fails via error action.Actual trace output:
{ "final_status": "success", "nodes": [{"node_id": "test-llm", "success": true}] }Expected trace output:
{ "final_status": "failed", "nodes": [{"node_id": "test-llm", "success": false}] }Root Cause: In
src/pflow/runtime/instrumented_wrapper.py, trace is recorded withsuccess=Truebefore checking if the node returned "error" action. The caching logic already checksresult != "error"but trace recording doesn't.Impact
Environment