Added support to Eager CodeGen for multiple in-place parameters. #10945
Conversation
| @@ -1 +1,2 @@ | |||
| Tensor gemm(const Tensor& A, const Tensor& B, const Tensor& C, float alpha, float beta, int transA, int transB); | |||
| Tensor gemm(const Tensor& A, const Tensor& B, const Tensor& C, float alpha, float beta, int transA, int transB); | |||
| std::tuple<Tensor, Tensor, Tensor> batchnorm_inplace(Tensor& X, const Tensor& scale, const Tensor& B, Tensor& input_mean, Tensor& input_var, const float epsilon, const float momentum); // {"schema": "batchnorm_inplace(Tensor(a!) X, Tensor scale, Tensor b, Tensor(b!) input_mean, Tensor(c!) input_var, float epsilon, float momentum) -> (Tensor(a!), Tensor(b!), Tensor(c!))", "dispatch": "False", "default": "True"} | |||
There was a problem hiding this comment.
it doesn't look like an inplace op. typically the inplace signature return a Tensor reference instead of a tensor.
There was a problem hiding this comment.
You are right - I missed the reference. Updated.
| cpp_param = cpp_func.get_parameter(op_input) | ||
| if cpp_param: | ||
| for torch_p in cpp_param.torch_param: | ||
| if isinstance(return_info, ast.TupleType): |
There was a problem hiding this comment.
I am not sure whether it is good idea to use tuple in the return type. there is some type like tensor list or array used in Aten native function. could we use those types?
There was a problem hiding this comment.
All the native ATen functions which return multiple in-place ops use std::tuple. e.g.
::std::tuple<Tensor &,Tensor &> cummax_out(const Tensor & self, int64_t dim, Tensor & values, Tensor & indices); // {"schema": "aten::cummax.out(Tensor self, int dim, *, Tensor(a!) values, Tensor(b!) indices) -> (Tensor(a!) values, Tensor(b!) indices)", "dispatch": "True", "default": "True"}
…rosoft#10945) * Added support to CodeGen for multiple inplace output parameters. * Updated output Tensor to references.
…rosoft#10945) * Added support to CodeGen for multiple inplace output parameters. * Updated output Tensor to references.
Currently the ORT eager codegen only supports a single in-place parameters. This PR adds support for multiple in-place parameters. Additionally it adds a test custom op for batch normalization inline which uses this functionality.