Add shape inference design doc #3693
Conversation
wangkuiyi
requested review from
Superjomn,
reyoung,
jacquesqiao and
qingqing01
|
This PR summarizes the discussion a few hours ago. Anyone please feel free to complete it. |
paddle/framework/type_inference.md
Outdated
| LOD_TENSOR = 6; | ||
| } | ||
| Type type = 1; | ||
| optional LODTesnorDesc lod_tensor = 2; // when type==LOD_TENSOR |
There was a problem hiding this comment.
LOD_TENSOR just a description of tensor's sequence information. It could be float lod tensor,int lod tensor, etc.
There was a problem hiding this comment.
Yes. We have LODTensor::element_type.
|
|
||
| ```protobuf | ||
| message VarDesc { | ||
| enum Type { |
There was a problem hiding this comment.
Also, very strange for Variable Type. If it is variable, the type should be
- Tensor
- LODTensor
- StepScopes.
- Etc.
Not int/float. They are types for tensors.
There was a problem hiding this comment.
Our Variable is dynamic-typed, like a Python variable, which can hold any typed value. So VarDesc needs a field type.
A variable can hold a single int, so there should be a enum Type { INT;.
A tensor can hold any typed elements, so TensorDesc needs a field element_type.
I am changing the text in accordance.
paddle/framework/type_inference.md
Outdated
| } | ||
|
|
||
| message LODTensorDesc { | ||
| repeated int dims = 1; // [UNK, UNK, 6000] is saved as [-1, -1, 6000] |
There was a problem hiding this comment.
saved as [0, 0, 6000] is better. If unk is the short for batch-size.
There was a problem hiding this comment.
0 is a valid dim size. Consider that IfOp needs to split the input for its left and right branches. It could be that the input for the left or right branch happens to have 0 instances.
| ```c++ | ||
| class Block : public OperatorBase { | ||
| private: | ||
| map<string, VarDesc> vars_; |
There was a problem hiding this comment.
The shape information is stored in paddle::framework::Tensor. It is no need to store it again in in VarDesc.
| ```c++ | ||
| class OperatorBase { | ||
| private: | ||
| const OpDesc& desc_; |
There was a problem hiding this comment.
There are many operators not created by OpDesc in backward, optimization. It is no need to store OpDesc, and not able to store OpDesc.
| InferShape(block); | ||
| } | ||
| private: | ||
| virtual InferShape(Block* block) = 0; |
There was a problem hiding this comment.
Why should InferShape be redesigned again?
|
Closing this too old PR. |
Variable definition and type inference belong to Block (used to be known as NetOp) instead of Scope.