Summary
The flow engine would benefit from a notion of a canonical final output step, plus validation/linting around pre-final result consumption.
Problem
In non-trivial flows there is often a progression like:
buildResult
- optional enrich/merge steps
- persistence or upload steps
finalizeResult
Today, parent flows and cache steps can accidentally read an intermediate output instead of the finalized one. This is especially easy when the flow technically runs, but the consumer misses persisted URLs, normalized fields, or merged data that only exist after finalization.
Requested behavior
A few options could help:
- support a canonical output step concept, inferred from
finalizeResult or declared explicitly in flow metadata
- warn when
buildCacheEntry caches a pre-final result even though a canonical final output exists
- warn when a parent flow reads a child flow's intermediate step output instead of the canonical final output
Why this would help
This would catch bugs that are otherwise very subtle:
- cached results missing persisted fields
- parent flows consuming incomplete child data
- output shape drift between fresh runs and cached runs
Possible design direction
Even a lightweight lint would be valuable. For example:
- default canonical step order:
finalizeResult > buildResult
- optional flow metadata override such as
canonicalOutputStep
- validator warnings first, hard errors later if the heuristic proves reliable
This seems like a good fit for flow validate because the engine already knows step ids and execution order.
Summary
The flow engine would benefit from a notion of a canonical final output step, plus validation/linting around pre-final result consumption.
Problem
In non-trivial flows there is often a progression like:
buildResultfinalizeResultToday, parent flows and cache steps can accidentally read an intermediate output instead of the finalized one. This is especially easy when the flow technically runs, but the consumer misses persisted URLs, normalized fields, or merged data that only exist after finalization.
Requested behavior
A few options could help:
finalizeResultor declared explicitly in flow metadatabuildCacheEntrycaches a pre-final result even though a canonical final output existsWhy this would help
This would catch bugs that are otherwise very subtle:
Possible design direction
Even a lightweight lint would be valuable. For example:
finalizeResult>buildResultcanonicalOutputStepThis seems like a good fit for
flow validatebecause the engine already knows step ids and execution order.