You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
PR fixing GH #333 (sort_keys cache bug) also addressed the heterogeneous batch sub-workflow planner gap — batch nodes with workflow: ${item.workflow} (like emotional-reviews in lyrics-generator) were marked opaque because the planner couldn't resolve per-item workflow paths.
The fix adds per-item workflow resolution to the planner, which mirrors logic already in the runtime's WorkflowExecutor (compile-once cache, per-item dispatch). This is intentional duplication — tracked here for cleanup.
_compiled_workflow_cache (class-level dict keyed by resolved path)
_loaded_ir_cache (class-level dict keyed by raw workflow ref)
Per-item dispatch happens in exec() via the batch executor callback
What's shared: The per-item template resolution → workflow path → compile → cache pattern. The planner reimplements this because it can't call WorkflowExecutor.exec() (no side effects allowed).
A shared abstraction that both the runtime and planner call. Candidate shape:
# Shared: resolve batch items to per-item workflow targetsdefresolve_heterogeneous_batch_workflows(
items: list[Any],
workflow_ref_template: str,
item_alias: str,
template_config: TemplateConfig,
shared: dict,
base_path: Path,
registry: Registry,
) ->dict[str, CompiledWorkflow]:
"""Resolve and compile unique child workflows for a heterogeneous batch. Returns a dict keyed by resolved path → compiled workflow. """
The runtime's WorkflowExecutor and the planner's _plan_heterogeneous_batch_items would both call this instead of reimplementing the resolve → compile → cache loop.
Where the Code Lives
Planner heterogeneous path: src/pflow/execution/plan.py::_plan_heterogeneous_batch_items (search for compile_cache)
Planner dispatch: src/pflow/execution/plan.py::_plan_batch_sub_workflow — branches on is_per_item_workflow
No users yet. Correct dry-run behavior (showing per-item cache status for heterogeneous batches) is more valuable now than architectural purity. The duplication is localized (~70 LOC in one function) and tracked here.
Acceptance Criteria
Shared primitive extracted and used by both planner and runtime
make test && make check clean
test_plan_drift.py (30+ cases) passes unchanged
No behavioral change — same dry-run output, same runtime behavior
Context
PR fixing GH #333 (sort_keys cache bug) also addressed the heterogeneous batch sub-workflow planner gap — batch nodes with
workflow: ${item.workflow}(likeemotional-reviewsin lyrics-generator) were marked opaque because the planner couldn't resolve per-item workflow paths.The fix adds per-item workflow resolution to the planner, which mirrors logic already in the runtime's
WorkflowExecutor(compile-once cache, per-item dispatch). This is intentional duplication — tracked here for cleanup.What's Duplicated
1. Per-item workflow resolution + compile cache
Planner:
src/pflow/execution/plan.py::_plan_heterogeneous_batch_items(~70 LOC)compile_cache: dict[str, _PreparedSubWorkflow])_build_plan_with_sharedRuntime:
src/pflow/runtime/workflow_executor.py::WorkflowExecutor_compiled_workflow_cache(class-level dict keyed by resolved path)_loaded_ir_cache(class-level dict keyed by raw workflow ref)exec()via the batch executor callbackWhat's shared: The per-item template resolution → workflow path → compile → cache pattern. The planner reimplements this because it can't call
WorkflowExecutor.exec()(no side effects allowed).2. Output population (#321 item A)
Planner:
plan.py::_extract_child_outputs→_resolve_declared_outputs+_mirror_child_shared(~30 LOC)Runtime:
workflow_executor.py::_expose_child_outputs(~27 LOC)Already documented in #321.
3. Cycle detection (#321 item B)
Planner:
plan.pyusesvisited_pathsargument threadingRuntime:
workflow_executor.pyusesshared["_pflow_stack"]listAlready documented in #321.
What to Extract
A shared abstraction that both the runtime and planner call. Candidate shape:
The runtime's
WorkflowExecutorand the planner's_plan_heterogeneous_batch_itemswould both call this instead of reimplementing the resolve → compile → cache loop.Where the Code Lives
src/pflow/execution/plan.py::_plan_heterogeneous_batch_items(search forcompile_cache)src/pflow/execution/plan.py::_plan_batch_sub_workflow— branches onis_per_item_workflowsrc/pflow/runtime/workflow_executor.py—_compiled_workflow_cache,_loaded_ir_cachesrc/pflow/runtime/workflow_executor.py::exec()— resolved inside the batch executor callbackWorkflowExecutor.is_exposable_child_key,WorkflowExecutor.MAX_DEPTH_DEFAULT,is_clean_termination(from PR feat: --dry-run with plan + cost/duration estimates (fixes #310) #320)Why It Was Shipped as Duplication
No users yet. Correct dry-run behavior (showing per-item cache status for heterogeneous batches) is more valuable now than architectural purity. The duplication is localized (~70 LOC in one function) and tracked here.
Acceptance Criteria
make test && make checkcleantest_plan_drift.py(30+ cases) passes unchangedRelated