Add Intermediate Kernel API for refactor Tensor Lib #36914

YuanRisheng · 2021-11-01T12:55:03Z

PR types

Others

PR changes

Others

Describe

Add Intermediate Kernel APIs which encapsulate infershape function and lower-level Kernel API.It makes kernel Call more convenient.
For Example，you can write following code when you want to call a "dot" in a new kernel:
auto out = Dot<T>(dev_ctx, x, y);

… op2func_refactor

* replace to small vector and change to const & * add std::move Co-authored-by: Chen Weihang <chenweihang@baidu.com>

… op2func_refactor

…into flatten_refactor

* add a candidate dense tensor class, test=develop * remove TensorBase::backend(), test=develop * remove some ops, test=develop * cherry-pick the pr of tensor meta, test=develop * moves the dense tensor and some ops, test=develop * update the linalg operator, test=develop * update other operators, test=develop * fix errors, test=develop * fix bugs, test=develop * try to resolve the problem of windows ci, test=develop * updates codes, test=develop * fix the tensor_utils.cc, test=develop * modify the dense tensor, test=develop * fix the data type, test=develop Co-authored-by: shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>

…into op2func_refactor

paddle-bot-old · 2021-11-01T12:55:06Z

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

chenwhql · 2021-11-02T03:18:42Z

paddle/fluid/framework/operator.cc

@@ -23,6 +23,7 @@ limitations under the License. */
 #include "paddle/fluid/framework/data_type_transform.h"
 #include "paddle/fluid/framework/details/nan_inf_utils.h"
 #include "paddle/fluid/framework/op_call_stack.h"
+#include "paddle/fluid/framework/pten_utils.h"


可以移除的头文件，operator.h中已有

Shixiaowei02 · 2021-11-02T03:19:42Z

paddle/pten/api/include/creation.h

+                        const DenseTensor& x,
+                        const Scalar& val) {
+  auto out_meta = UnchangedInferShape(x.meta());
+  const auto allocator =


[TODO] 这里后续我改为全局单例，减小开销

chenwhql · 2021-11-02T03:19:02Z

paddle/fluid/framework/operator.cc

@@ -23,6 +23,7 @@ limitations under the License. */
 #include "paddle/fluid/framework/data_type_transform.h"
 #include "paddle/fluid/framework/details/nan_inf_utils.h"
 #include "paddle/fluid/framework/op_call_stack.h"
+#include "paddle/fluid/framework/pten_utils.h"


可以移除的头文件，operator.h中已有

chenwhql · 2021-11-02T03:19:05Z

paddle/fluid/imperative/prepared_operator.cc

@@ -16,6 +16,7 @@

 #include "paddle/fluid/framework/data_type_transform.h"
 #include "paddle/fluid/framework/details/nan_inf_utils.h"
+#include "paddle/fluid/framework/pten_utils.h"


chenwhql · 2021-11-02T03:19:54Z

paddle/pten/api/include/creation.h

+namespace pten {
+
+template <typename T, typename ContextT>
+DenseTensor FillAnyLike(const ContextT& dev_ctx,


[TODO] 后续我们需要解决这个问题，API和Kernel函数命名尽可能一致，不参照原先op的命名

chenwhql · 2021-11-02T03:21:01Z

paddle/pten/api/include/math.h

+
+namespace pten {
+
+template <typename T, typename ContextT>


[TODO] 后续这里的代码也需要自动生成，需要综合考虑 @zyfncg

chenwhql · 2021-11-02T03:22:32Z

paddle/pten/tests/test_dot_api.cc

@@ -82,3 +84,53 @@ TEST(API, dot) {
  ASSERT_NEAR(expect_result[1], actual_result1, 1e-6f);
  ASSERT_NEAR(expect_result[2], actual_result2, 1e-6f);
 }
+
+TEST(DEV_API, dot) {


[TODO] 外部API的单测和内部API的单测需要分开管理，便于编译解耦

…refactor

chenwhql

LGTM

* initial tensor design & sign kernel demo * add move constructor for meta & add lodtensor * add dirs & sign xpu kernel * add mean cpu&cuda kernel impl * move sign & mean xpu & npu kernel * add selected_rows basic impl * refactor design, BaseTensor to DenseTensor, etc. * add scale mkldnn kernel * polish xpu & npu impl details * fix mkldnn reuse compile failed * change tensor operation lib name * rename util filename * add more comments * change TensorImplInterface to TensorInterface * add kernel key and factory * remove MKLDNNTensorMeta, add MKLDNNDenseTensor * change XXDeviceContext to XXContext * add base kernel registrar utils & test on sign * replace boost::any by paddle::any * fix several ci failed * fix npu compile error * add ordered map util * fix multiple ordered_map compile errors * move dev into include dir * support sign op in static op run * fix static op run error * fix new executor compile failed * add dygraph branch & remove sign_op.h * fix test_infer_no_need_buffer_slots * fix rocm compile link error * fix unitybuild error & clear glog * fix npu compile failed * skip quant trans test * fix part windows compile problem * fix xpu enforce error * fix inference test failed * remove ordered_map to solve quant failed * fix part of rcom compile faild * add more register kernels * revert scale kernel temporarily * fix code format error * add new kernel registrar marco * rename top to tcmpt * revert xpu, npu, mkldnn impl & remove op def * add kernel args parse functor to auto parse args * revert some change & add scale kernels * add op proto in dygraph kernelcontext building * polish kernel dispatch logic & nameing rule * fix scale kernel match error * fix scale test failed * add mean API and unittest * test mean api success * add branch to solve compiled error * skip clang format error * add mean skip rule in op_library * add dot kernel, api and unittest (PaddlePaddle#6) * remove old kernel and add symbol link * fix dot compiled failed * add merco for module declare * fix npu and xpu compile error * revert sign, mean, scale, dot kernel removing * add comment for keeping old kernel impl * fix mutable_data error * fix bfloat16 conflit * fix inference undef error * adapt to msvc compile rules * polish comment for template inst * add cmake template instantiation for win * fix backend to place device id bug * fix ifdef error * Op2functor (PaddlePaddle#7) * add kernel args maker class * make args maker non-const * remove debug log * modify codes by review options * split constructPrKernelContext function * fix output name bug * fix test_mean_op test_sign_op failed * fill_any_like kernel refactor (PaddlePaddle#10) * fill_any_like kernel refactor * remove useless code of full_like c++ api * skip dtype for fill_any_like * add attrs for kernel key constrcut * add use_pt_kernel Flags to control whether to use pt kernel (PaddlePaddle#13) * add use_pt_kernel Flags to control whether to use pt kernel * change the default value to true for cheking pt kernels * fix mutable_data cuda place error * move high level apis into hapi * remove selectedrows adapting temporarily * Support Scalar in Tensor Compute Library (PaddlePaddle#14) * fill_any_like kernel refactor * remove useless code of full_like c++ api * Support Scalar in Tensor Compute Library * add scalar in dygraph and static graph mode * keep the basic type for attr, instead of using scalar for all * merge the code * remove mkldnn tensor & polish details * use flat_hash_map and small_vector in kernel factory * Refactor flatten kernel (PaddlePaddle#12) * refactor flatten kernel * update infershape function * fix compile bugs * fix bugs when merge * fix compiler bugs * fix bugs when run test_flatten_api * fix bugs when run test * Revert "use flat_hash_map and small_vector in kernel factory" This reverts commit 2309149. * Move cpu, cuda and other device code into kernels (PaddlePaddle#15) * fill_any_like kernel refactor * remove useless code of full_like c++ api * Support Scalar in Tensor Compute Library * add scalar in dygraph and static graph mode * keep the basic type for attr, instead of using scalar for all * merge the code * start refactor matmul * move cpu, cuda and other device modules into kernels * merge code * polish code in operator.cc * Perfect unitests (PaddlePaddle#16) * perfect unittest * update license * replace with flat_hash_map, small_vector (PaddlePaddle#19) * fix small_vector build error on windows platform * replace with flat_hash_map, small_vector * remove todo * Perfect unitests (PaddlePaddle#20) * perfect unittest * update license * fix bug when run tcmpt_utils_test * refactor execution adapting impl * fix insert conflit * Fix CI bug of test_yolov3 (PaddlePaddle#21) * fill_any_like kernel refactor * remove useless code of full_like c++ api * Support Scalar in Tensor Compute Library * add scalar in dygraph and static graph mode * keep the basic type for attr, instead of using scalar for all * merge the code * start refactor matmul * move cpu, cuda and other device modules into kernels * merge code * polish code in operator.cc * Fix CI bug of test_yolov3 * add the tensor base class, test=develop (PaddlePaddle#17) * update the tensor base class, test=develop * remove two funcs, test=develop * update the error msg, test=develop Co-authored-by: Chen Weihang <chenweihang@baidu.com> * [no-verify] commit backend and tensor signature changes * Rename tcmpt to pten (PaddlePaddle#23) * rename tcmpt to pten * update omitted files for rename to pten * update omitted file for rename to pten * remove k of all enum var * remove kernel_instantiate (PaddlePaddle#26) * remove symbols and spatial_tensor * change common to functions * readd share tensor impl methods * add a candidate dense tensor class, test=develop (PaddlePaddle#28) * change all Pt to Pten * resolve conflit with xiaowei * Op2functor opt1 (PaddlePaddle#27) * replace to small vector and change to const & * add std::move Co-authored-by: Chen Weihang <chenweihang@baidu.com> * polish kernel factory and kernel registry * fix operator test error msg mismatch * remove tensor signature and backend set member * move scalar and polish enforce * revert dtype layout change to fix error * fix enum operator override error * Add Intermediate API layer * add several base unittests * add pten utils tests * polish some details * Dev/op2func refactor 3 (PaddlePaddle#30) * add a candidate dense tensor class, test=develop * remove TensorBase::backend(), test=develop * remove some ops, test=develop * cherry-pick the pr of tensor meta, test=develop * moves the dense tensor and some ops, test=develop * update the linalg operator, test=develop * update other operators, test=develop * fix errors, test=develop * fix bugs, test=develop * try to resolve the problem of windows ci, test=develop * updates codes, test=develop * fix the tensor_utils.cc, test=develop * modify the dense tensor, test=develop * fix the data type, test=develop Co-authored-by: shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com> * intermediate api adapt to new dense tensor * add some TODO and delete include header Co-authored-by: Chen Weihang <chenweihang@baidu.com> Co-authored-by: chentianyu03 <ctychentianyu@gmail.com> Co-authored-by: zyfncg <1370305206@qq.com> Co-authored-by: 石晓伟 <39303645+Shixiaowei02@users.noreply.github.com>

chenwhql added 30 commits July 9, 2021 11:50

initial tensor design & sign kernel demo

3f545c4

add move constructor for meta & add lodtensor

1f4ea40

add dirs & sign xpu kernel

44bf926

add mean cpu&cuda kernel impl

b20689d

move sign & mean xpu & npu kernel

79d2a1a

add selected_rows basic impl

434136f

refactor design, BaseTensor to DenseTensor, etc.

6c6ee22

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

013c3fb

… op2func_refactor

add scale mkldnn kernel

33bba06

polish xpu & npu impl details

d895a11

fix mkldnn reuse compile failed

62ebf01

change tensor operation lib name

7c09726

resolve conflit with develop

7ae7f2f

rename util filename

288efc2

add more comments

be3ddd5

change TensorImplInterface to TensorInterface

3386c49

add kernel key and factory

4ef6be5

remove MKLDNNTensorMeta, add MKLDNNDenseTensor

b69066e

resolve conflict with develop

1d4f90e

change XXDeviceContext to XXContext

c732d57

add base kernel registrar utils & test on sign

374345f

resolve conflict with develop

bbb6473

replace boost::any by paddle::any

0e18ff4

fix several ci failed

805896b

fix npu compile error

fc4442b

add ordered map util

cefe30a

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

aa3e79b

… op2func_refactor

fix multiple ordered_map compile errors

a1753a0

move dev into include dir

05a82e7

support sign op in static op run

90e9090

MingMingShangTian and others added 18 commits October 21, 2021 10:38

Op2functor opt1 (PaddlePaddle#27)

76a588e

* replace to small vector and change to const & * add std::move Co-authored-by: Chen Weihang <chenweihang@baidu.com>

polish kernel factory and kernel registry

fb224ab

fix operator test error msg mismatch

252fb79

remove tensor signature and backend set member

19b1095

move scalar and polish enforce

24ef6c5

revert dtype layout change to fix error

1685b67

fix enum operator override error

7b7e988

Add Intermediate API layer

7c41b15

add several base unittests

52fead0

add pten utils tests

2ff2721

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

e3ed2c6

… op2func_refactor

polish some details

b5c77e5

Merge branch 'op2func_refactor' of https://github.com/chenwhql/Paddle …

471ae40

…into flatten_refactor

Merge branch 'op2func_refactor' of https://github.com/chenwhql/Paddle …

5fb285c

…into op2func_refactor

Merge from op2refactor

e2731a0

intermediate api adapt to new dense tensor

16e6bf1

Merge From Develop

dfec7a0

chenwhql reviewed Nov 2, 2021

View reviewed changes

Shixiaowei02 reviewed Nov 2, 2021

View reviewed changes

chenwhql reviewed Nov 2, 2021

View reviewed changes

YuanRisheng added 2 commits November 2, 2021 03:45

Merge branch 'develop' of github.com:YuanRisheng/Paddle into flatten_…

3b2e950

…refactor

add some TODO and delete include header

aa8a3dd

chenwhql approved these changes Nov 2, 2021

View reviewed changes

zyfncg approved these changes Nov 2, 2021

View reviewed changes

Shixiaowei02 approved these changes Nov 2, 2021

View reviewed changes

chenwhql merged commit 4a7f1a0 into PaddlePaddle:develop Nov 2, 2021

YuanRisheng deleted the flatten_refactor_to_develop branch November 19, 2021 02:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Intermediate Kernel API for refactor Tensor Lib #36914

Add Intermediate Kernel API for refactor Tensor Lib #36914

YuanRisheng commented Nov 1, 2021 •

edited by chenwhql

Loading

paddle-bot-old bot commented Nov 1, 2021

chenwhql Nov 2, 2021

YuanRisheng Nov 2, 2021

Shixiaowei02 Nov 2, 2021

chenwhql Nov 2, 2021

YuanRisheng Nov 2, 2021

chenwhql Nov 2, 2021

chenwhql Nov 2, 2021

chenwhql Nov 2, 2021

chenwhql Nov 2, 2021 •

edited

Loading

chenwhql left a comment

Add Intermediate Kernel API for refactor Tensor Lib #36914

Add Intermediate Kernel API for refactor Tensor Lib #36914

Conversation

YuanRisheng commented Nov 1, 2021 • edited by chenwhql Loading

PR types

PR changes

Describe

paddle-bot-old bot commented Nov 1, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chenwhql Nov 2, 2021 • edited Loading

Choose a reason for hiding this comment

chenwhql left a comment

Choose a reason for hiding this comment

YuanRisheng commented Nov 1, 2021 •

edited by chenwhql

Loading

chenwhql Nov 2, 2021 •

edited

Loading