What happened?
amber/src/main/scala/org/apache/texera/amber/engine/architecture/scheduling/config/LinkConfig.scala::LinkConfig.toPartitioning builds the channel list for the BroadcastPartition arm with fromWorkerIds.zip(toWorkerIds) — i.e. 1:1 pairing — instead of the full cross product. Two issues follow:
- The semantics of broadcast are "every sender targets every receiver", which a zip pairing does not encode.
- The companion helper
ChannelConfig.generateChannelConfigs in the same package emits a full cross product for BroadcastPartition. The two helpers are expected to agree on routing; today they diverge for this arm.
For unequal-length input lists List.zip also silently drops the surplus side, parallel to issue #4799 for OneToOnePartition.
How to reproduce?
import org.apache.texera.amber.core.virtualidentity.ActorVirtualIdentity
import org.apache.texera.amber.core.workflow.BroadcastPartition
import org.apache.texera.amber.engine.architecture.scheduling.config.LinkConfig
import org.apache.texera.amber.engine.architecture.sendsemantics.partitionings.BroadcastPartitioning
val w = (1 to 3).map(i => ActorVirtualIdentity(s"w$i")).toList
val u = (1 to 3).map(i => ActorVirtualIdentity(s"u$i")).toList
val out = LinkConfig.toPartitioning(w, u, BroadcastPartition(), 64)
val bp = out.asInstanceOf[BroadcastPartitioning]
bp.channels.map(c => (c.fromWorkerId.name, c.toWorkerId.name))
// Returns: ((w1,u1), (w2,u2), (w3,u3))
// Broadcast semantics would predict the full 3*3 cross product.
Version
1.1.0-incubating (Pre-release/Master)
What happened?
amber/src/main/scala/org/apache/texera/amber/engine/architecture/scheduling/config/LinkConfig.scala::LinkConfig.toPartitioningbuilds the channel list for theBroadcastPartitionarm withfromWorkerIds.zip(toWorkerIds)— i.e. 1:1 pairing — instead of the full cross product. Two issues follow:ChannelConfig.generateChannelConfigsin the same package emits a full cross product forBroadcastPartition. The two helpers are expected to agree on routing; today they diverge for this arm.For unequal-length input lists
List.zipalso silently drops the surplus side, parallel to issue #4799 forOneToOnePartition.How to reproduce?
Version
1.1.0-incubating (Pre-release/Master)