Introduce Loop

[johnyangk]

Introduce the concept of `Loop' in our system. Need following considerations.

• Frontend(Beam): Identify loops in Beam programs
• IR: How shall we express loop entry/exit? What about conditional loops? Might be a good idea to refer to the Naiad paper.
• Backend: How shall we execute the loop?

[johnyangk]

Let's also do some research on expressing 'Loop' with Beam.
For example, the recently added stateful processing might be useful: https://beam.apache.org/blog/2017/02/13/stateful-processing.html

[johnyangk]

I quickly skimmed through the post on stateful processing, and it seems to be more of a replacement for sideInput than a primitive for looping. It seems that we still can only express acyclic graphs with Beam.

[wonook]

I decided to handle this issue with a ConditionalOperator? LoopOperator?, as handled with Musketeer IR.

[johnyangk]

Thanks! Please provide a detailed description of the design in the PR message for those of us who are not familiar with Musketeer. 😄

[wonook]

Sure, it'll soon arrive with the PR.

[wonook]

I'll handle this through creating two passes: LoopRollingPass and LoopUnrollingPass, to create loops where applicable, as well as to unroll loops to test and modify DAGs flexibly.
They will use the LoopOperator, which includes the number of iterations, for the expression.