[FLINK-2897] [runtime] Use distinct initial indices for OutputEmitter round-robin #1292
Conversation
There was a problem hiding this comment.
This method preserves the behavior of FORWARD only if numberOfChannels is always 1 when FORWARD is configured which is out of control of the OutputEmitter.
Should we add a check somewhere (e.g., in this condition to reduce the overhead) to ensure that numberOfChannels is acutally 1 for FORWARD?
There was a problem hiding this comment.
Yes, I did overlook that, though customPartition does likewise and the OutputEmitter will only be used for a single strategy. The strategy is set in the constructor so the channel list can be created there for the types which do not broadcast.
greghogan
force-pushed
the
2897_use_distinct_initial_indices_for_outputemitter_round_robin
branch
from
6365e69 to
699009f
Compare
There was a problem hiding this comment.
Shouldn't this case be also covered by nextChannel %= numberOfChannels?
There was a problem hiding this comment.
The modulo computation is executed in two cases: 1) the initial call to robin when the source parallelism is greater than the receiver parallelism and 2) when the receiver parallelism is dynamically changing.
Unoptimized modulo is much slower than a test for equality but is required in the first case to evenly distribute the first outputs.
|
Aside from till's comment (simplification of the "robin()" case), this looks good. |
|
Yes, +1 for merging. Thanks for your contribution @greghogan. |
|
Will merge this... |
… round-robin This closes apache#1292
5 participants
No description provided.