-
Notifications
You must be signed in to change notification settings - Fork 2.8k
Graph
-
Node (graph.h/.cc)
-
What's the difference between an Op and a Node?
-
What are the common properties for a node?
- Can a node's name be empty?
- What's the identifier of a node in a graph? Index or Name?
-
Advance: Function Ops, node with FunctionBody
-
-
Graph (graph.h/.cc)
-
How to traverse from one node to another node?
-
What's the difference between a GraphInput and an Initializer?
-
Look for an example using GetProducerNode() and GetConsumerNodes()
-
What's the purpose of Graph::Resolve()?
- How is ShapeAndTypeInference invoked?
-
-
NodeArg (node_arg.h)
- What the relationship between a graph edge and a NodeArg?
- What's the unique identifier of a NodeArg in a graph?
- Action: Look for some example using Graph::GetOrCreateNodeArg() (You will need to use this at some point)
-
Understand the difference between GraphTransformer and RewriteRule
-
Understanding the purpose of GraphTransformerManager
- How to register a set of graph transformers into a session?
-
Understanding the two versions of graph_transformer_utils.cc (onnxruntime/orttraining ones)
-
Get familiar with graph_utils.cc
-
Experiment with onnx.helper to compose a onnx model from the script (see transpose_matmul_gen.py for examples)
-
Action: Implement a graph transformer to get hands-on experience
-
Understand the workflow of training graph transformation
-
Understand GraphAugmenter (graph_augmenter.h/.cc)
-
GradientGraphBuilder
- Understand the purpose/usage of STOP_GRADIENT_EDGES
- Understand the meaning of x_node_args/y_node_args
- Advance: Understand the back-propagation process in GradientGraphBuilder::Build()
-
Per Op GradientBuilder
-
Understand the Gradient Registry (gradient_builder_registry.cc)
-
Understand the Gradient Builder Declaration (gradient_builder.h)
-
Read a few examples in Gradient Builder Implementation (gradient_builder.cc)
-
Understand the shorthands of I, GI, O, GO (gradient_builder_base.h)
-
Understand how gradient subgraph is composed with existing ops, followings are good examples
- Easy: GetDropoutGradient, GetSqrtGradient
- Medium: GetAddSubGradient, GetMulGradient
- Hard: GetMatMulGradient, GetGemmGradient
-
Understand how broadcasting is handled when building gradient graph GradientBuilderBase::HandleBroadcasting()
-
Action: Implement a gradient definition for an op to get hands-on experience
-
Good reference implementation: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/ops/array_grad.py
-
-
Please use the learning roadmap on the home wiki page for building general understanding of ORT.