[PaddlePaddle Hackathon 4][Frontend][Paddle]add thresholded_relu/index_select/eye/linspace/take_alone_axis/dist for paddle frontend #14172
Conversation
|
Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment.
Generated by tvm-bot |
…_alone_axis/dist for paddle frontend
|
|
||
| x = g.get_node(op.input("X")[0]) | ||
| y = g.get_node(op.input("Y")[0]) | ||
| z = _op.abs(_op.subtract(x, y)) |
There was a problem hiding this comment.
z = x - y. It can't use _op.abs, otherwise inv_p=_expr.const(1.0 / p, dtype=dtype) miscalculated.
There was a problem hiding this comment.
I am confused that z would lead to the miscalculation of inv_p. I refer to the code https://github.com/PaddlePaddle/Paddle2ONNX/blob/develop/paddle2onnx/mapper/tensor/dist.cc#L33 and api doc
There was a problem hiding this comment.
Sorry, I saw p as z.
No problem here, but _op.abs(z) should no longer be used in the following code.
| stop = g.get_node(op.input("Stop")[0]) | ||
| num = g.get_node(op.input("Num")[0]) | ||
| dtype = _convert_dtype_value(op.attr("dtype")) | ||
| start, infered = try_infer_value(start, parameters=g.get_params()) |
There was a problem hiding this comment.
Dynamic shape should also be supported
| if num == 1: | ||
| out = _op.full(_expr.const(start, dtype), shape=(1)) | ||
| else: | ||
| if dtype in ["int32", "int64"]: |
There was a problem hiding this comment.
dtype support float32, float64, int32 and int64
| start = _expr.const(start, "float32") | ||
| stop = _expr.const(stop, "float32") | ||
| step = _expr.const(step, "float32") | ||
| out = _op.transform.arange(start=start, stop=stop, step=step, dtype="float32") |
There was a problem hiding this comment.
Why is dtype fixed to "float32"?
| """Operator converter for eye.""" | ||
|
|
||
| num_rows = op.attr("num_rows") | ||
| num_columns = op.attr("num_columns") |
There was a problem hiding this comment.
num_columns might equal -1, in which case num_columns equals num_rows.
There was a problem hiding this comment.
Refer to the code https://github.com/PaddlePaddle/Paddle/blob/release/2.4/python/paddle/tensor/creation.py#L808, num_columns should be a non-negative int
There was a problem hiding this comment.
We should look at the c++ definition and implementation of op rather than the api.
The Paddle to TVM conversion is an op mapping rather than an api.
https://github.com/PaddlePaddle/Paddle/blob/b780a3ff4f89f73f4efd095002b320a5fe673afe/paddle/phi/kernels/impl/eye_kernel_impl.h#L44
| class ThresholdedRelu(nn.Layer): | ||
| @paddle.jit.to_static | ||
| def forward(self, inputs): | ||
| return nn.functional.thresholded_relu(inputs) |
There was a problem hiding this comment.
Add some cases where the threshold is 0.5, paddle.randn ranges from 0 to 1
| def forward(self, inputs): | ||
| return paddle.eye(3, 5, dtype="int32"), paddle.eye(3, 5, dtype="float32"), inputs | ||
|
|
||
| class Eye2(nn.Layer): |
There was a problem hiding this comment.
Add case where num_columns is None
|
Please update the PR to resolve the merge conflict :-) Happy to get it in afterwards |
| tmp_dtype = "float32" | ||
| else: | ||
| tmp_dtype = "float64" | ||
| start = _op.cast(start, tmp_dtype) |
There was a problem hiding this comment.
There should be static and dynamic shape cases.
If it is a static shape, the start value can be inferred, the op to TVM is very small;
If the value cannot be inferred from the dynamic shape, more op will be converted to TVM, which may affect performance.
There was a problem hiding this comment.
Sorry. I'm a bit confused. Even if the start value can be inferred, the output is still dynamic. How do we optimize it?
There was a problem hiding this comment.
if the start,stop and num value can be inferred, that's what you implemented before.
There is no need to add op like where, subtract, divide, etc
There was a problem hiding this comment.
Yes, but I think tvm will optimize automatically. If the start stop and num value can be inferred, these ops will be eliminated by FoldConstant pass in relay, it only outputs a relay.Constant. https://github.com/apache/tvm/blob/main/python/tvm/relay/frontend/paddlepaddle.py#L2572
There was a problem hiding this comment.
I tested the case where the values can be infered, the relay IR as follow.
def @main() {
meta[relay.Constant][0]
}
Add thresholded_relu/index_select/eye/linspace/take_alone_axis/dist for paddle frontend.
But in paddle 2.1.3, eye/linspace/take_alone_axis are not supported.
The test case has passed completely in version 2.4.2.