Repository navigation
Control blocks in Texera: design and roadmap #9263
Replies: 2 comments 3 replies
|
We’re discussing a use case with end users who want to run the same workflow with different input configurations. For example, they might provide a list of parameter sets or try every combination of selected values. Could we extend Form View and/or loops to support this? Users could enter their configurations then run the workflow for each configuration. How should we organize these executions? Should each configuration create a separate workflow run, or should all configurations run as iterations within a single run? With a loop, users would need a clear way to see which inputs produced which results. Could we show each iteration’s input values, status, outputs, and errors, and let users retry a failed iteration? What are the tradeoffs between these approaches, and how could this fit into the control-block roadmap? This would be valuable for the use case we’re exploring with end users. |
|
@sshiv012 If the use case is specifically related to "Form View," I suggest we start a new discussion, as the current discussion is about "Control block." |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Loops are now on Texera's main branch (#5700). Each existing thread covers one detail, such as
$namein properties (#8643) or the control-variable port (#8915). This thread is for the big picture: what a control block is, where it stands, and what to build next.What and why
An operator normally runs once, with its settings fixed before the run. Some tasks need part of a workflow to run again with a new value each time, for example trying K = 2 to 10 to pick the best K, or processing every file in a folder.
A control block is that part of the workflow. A start operator and an end operator enclose a body of ordinary operators, and short Python code on the two decides how the body runs. Loops are the first kind.
flowchart LR D[Data] --> LS subgraph blk [Control block: a loop] LS["Loop Start<br/>K = 2"] --> A["Body operators<br/>a setting uses $K"] --> LE["Loop End<br/>K += 1<br/>while K <= 10"] LE -. next iteration .-> LS end LE --> R["Results of<br/>all iterations"]Key ideas
Kor a file path change between iterations. A property refers to one as$K, so existing operators work unchanged.Today each iteration runs the block again on new workers, and a workflow with a loop runs in Materialized mode.
Roadmap
$namein Java and Scala operators (feat(amber): bind $i loop-variable references in operators #8639, fix(amber): reject $i in settings the plan is built from #8750, feat(frontend): accept $variable references in loop blocks #8640)Feature details can stay in their own threads.
All reactions