Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Request for documentation: Loop implementation #4762
I am trying to understand the implementation of
However, I cannot find any documentation on the ops that comprise
I am finding myself confused about some of the following concepts:
So far I've gotten quite a bit by just reading the source code, but it's pretty hard to build yourself a good mental model by going completely bottom up without having any high level picture of how the entire thing is organized.
If there is no intention to document these things (which would be completely understandable) please feel free to close this issue (although I would definitely appreciate some help or pointers as to where I can figure out the high-level overview of looping implementation).
I have been working on a doc which we hope to turn into a technical paper. I will give you a copy when it is ready. In the meantime, feel free to send any specific questions to me.
WhileContext is at this point a python front-end concept to represent a while_loop. It can be nested to represent nested loops. Internally, a loop is a cyclic graph using five primitive ops: Enter, Exit, Switch, Merge, and NextIteration. A loop can be partitioned onto multiple machines/devices, which is done in core/graph/graph_partition.cc. Frame is a concept used in core/common_runtime/executor.cc, which manages the execution of a partition assigned to a device. Most of the complexity in the python front-end is on backprop of while_loop.
@yuanbyu Thanks, good to hear that a doc is being prepared.
So, here's what I understand so far. Perhaps you can correct any misunderstandings and also this will serve as a useful resource for the time being for anyone else who runs into this.
The description above is not quite complete but I think I probably understand enough for what I want to do.
I have written a doc and would be happy to share it with you.
For your questions: