add pad func #26106

littletomatodonkey · 2020-08-10T14:12:50Z

PR types

Function optimization

PR changes

APIs

Describe

Add pad op

paddle-bot-old · 2020-08-10T14:12:59Z

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

LDOUBLEV · 2020-08-11T07:58:22Z

python/paddle/nn/functional/common.py

+    """
+    data_format = data_format.upper()
+
+    input_dim = len(input.shape)


这里可以加个判断，当input的维度数和pad的元素数不一致时，raise ValueError，不然报错会到底层

pad为var的时候，shape可能为[1, -1]，此时无法在编译时拿到length，所以不太好判断~

LDOUBLEV · 2020-08-11T07:59:45Z

python/paddle/nn/functional/common.py

+            elif input_dim == 4:
+                pad = concat([pad, zeros((2, ), dtype="int32")], axis=0)
+                unsqueezed_dim = [1]
+                input = unsqueeze(input, axes=unsqueezed_dim)


这里也加一个
else ：
raise ValueError, "data_format should be in one of "
"[NCL, NCHW, NCDHW, NLC, NHWC, NDHWC] but got {}".format(
data_format)

LDOUBLEV · 2020-08-11T08:02:55Z

python/paddle/nn/layer/common.py

@@ -254,91 +254,299 @@ def forward(self, input):
        return out


-class Pad2D(layers.Layer):


之前的Pad2D API是不需要了吗

已经补充回来了，thx

LDOUBLEV · 2020-08-11T08:05:27Z

python/paddle/fluid/tests/unittests/test_pad3d_op.py

+        self.assertRaises(Exception, test_reflect_2)
+
+        self.assertRaises(Exception, test_reflect_3)
+


是否需要加个pad1d的测试函数

这个是测试pad1d的：TestPad1dClass

willthefrog · 2020-08-13T05:46:52Z

paddle/fluid/operators/pad3d_op.cc

+    for (int c = 0; c < channels; ++c) {
+      for (int out_d = 0; out_d < out_depth; ++out_d) {
+        for (int out_h = 0; out_h < out_height; ++out_h) {
+          for (int out_w = 0; out_w < out_width; ++out_w) {


use std::memcpy for innermost dim? i.e., copy w (the last stride) elements in one go.

willthefrog · 2020-08-13T05:51:53Z

paddle/fluid/operators/pad3d_op.cc

+}
+
+template <typename T>
+void Pad3DConstNDHWC(const T* in_data, const int num, const int channels,


can those two be combined? the only difference is the last stride (w vs w*c)

willthefrog · 2020-08-13T05:59:32Z

paddle/fluid/operators/pad3d_op.cc

+}
+
+template <typename T>
+void Pad3DReflectNCDHW(const T* in_data, const int num, const int channels,


function for different mode can be merged? suggestion:

mode should be a template parameter, add templated function to calculate the padding value for given coordinates (e.g., something like get_pad_value(x, y))

fill the padded part of the canvas with the above method

iterate over c -> h or (1 - h) and memcpy w or c * w for the inner part of the canvas

willthefrog · 2020-08-13T06:01:24Z

paddle/fluid/operators/pad3d_op.cc

+}
+
+template <typename T>
+void Pad3DGradConstNCDHW(T* d_in_data, const int num, const int channels,


merge grad functions as above.

willthefrog · 2020-08-13T06:04:12Z

paddle/fluid/operators/pad3d_op.cc

+  }
+}
+
+static inline void GetPaddings(int* paddings,


simply return const int pointer to tensor.data or vector.data aspads are read only?

willthefrog · 2020-08-13T06:12:22Z

paddle/fluid/operators/pad3d_op.cu

+using framework::Tensor;
+
+template <typename T>
+__global__ void Pad3DConstNCDHW(const int nthreads, const T* in_data,


merge functions as above

difference being the kernel should iterate over the padded canvas, if the current coords falls in padded part, use templated method to calculate the pad value, otherwise read from original tensor.

but keep the kernels for different data layout separated as it would affect coalesced data access

it seems that function template pointer can not work properly in cuda codes.

willthefrog · 2020-08-13T06:44:45Z

python/paddle/nn/functional/common.py

+            raise ValueError, "data_format should be in one of "
+            "[NCL, NCHW, NCDHW, NLC, NHWC, NDHWC] but got {}".format(
+                data_format)
+


add fast path for dygraph mode. check the guide line for detailed info.

willthefrog · 2020-08-13T06:47:03Z

python/paddle/nn/layer/common.py

+
+class Pad(layers.Layer):
+    """
+        :alias_main: paddle.nn.Pad


:alais annotation are no longer needed.

removed, thx

willthefrog · 2020-08-13T06:47:58Z

python/paddle/nn/layer/common.py

+        self._pad = pad
+        self._name = name
+
+    def forward(self, input):


input -> x

willthefrog · 2020-08-13T10:14:26Z

paddle/fluid/operators/pad3d_op.cu

+    in_h = min(in_h, 2 * in_height - in_h - 2);
+    in_w = min(in_w, 2 * in_width - in_w - 2);
+
+    platform::CudaAtomicAdd(


atomic add is only needed for the reflected part.

willthefrog · 2020-08-13T10:16:02Z

paddle/fluid/operators/pad3d_op.cu

+    const int in_h = min(in_height - 1, max(out_h - pad_top, 0));
+    const int in_w = min(in_width - 1, max(out_w - pad_left, 0));
+
+    platform::CudaAtomicAdd(


atomic add is only needed for the border.

dyning · 2020-08-13T10:16:15Z

python/paddle/fluid/tests/unittests/test_pad3d_op.py

+                self.assertTrue(np.allclose(output.numpy(), np_out))
+
+
+class TestPad1dClass(unittest.TestCase):


这个命名需要修改

dyning · 2020-08-13T10:20:01Z

python/paddle/fluid/tests/unittests/test_pad3d_op.py

+            self.assertTrue(np.allclose(output.numpy(), np_out))
+
+
+class TestPadAPI(unittest.TestCase):


函数名需要确认下

已统一修改

willthefrog · 2020-08-17T03:22:17Z

python/paddle/nn/layer/common.py

+    Code Examples:
+        .. code-block:: python
+            import paddle
+            import paddle.fluid as fluid


willthefrog · 2020-08-17T03:24:10Z

python/paddle/tensor/__init__.py

@@ -111,6 +111,7 @@
 from .math import elementwise_div  #DEFINE_ALIAS
 from .math import elementwise_floordiv  #DEFINE_ALIAS
 from .math import elementwise_max  #DEFINE_ALIAS
+from .math import elementwise_mul  #DEFINE_ALIAS


git conflict? rebase?

paddle/fluid/operators/pad3d_op.cc

willthefrog · 2020-08-17T03:40:24Z

python/paddle/fluid/tests/unittests/test_pad3d_op.py

+        with program_guard(Program(), Program()):
+            input_shape = (1, 2, 3, 4, 5)
+            pad = [1, 2, 1, 1, 3, 4]
+            mode = "constant"


only test "constant"?

add other modes unit test, thx

willthefrog · 2020-08-17T03:41:18Z

python/paddle/fluid/tests/unittests/test_pad3d_op.py

+                self.assertTrue(np.allclose(output.numpy(), np_out))
+
+
+class TestPad1dClass(unittest.TestCase):


what happens to 2d? also, why is a separate test class needed?

Add Test1d 2 and 3d api separately, thx

willthefrog · 2020-08-17T03:42:48Z

python/paddle/nn/functional/common.py

+
+
+def pad(x,
+        pad=[0, 0, 0, 0],


remove default value, pad should be required.

WuHaobo · 2020-08-17T04:25:25Z

python/paddle/nn/functional/common.py

+from ...fluid.layers import unsqueeze
+from ...fluid.layers import elementwise_mul
+from ...tensor import clamp
+from ...tensor import sum


#DEFINE_ALIAS 都加一下吧，文档映射那边会用

好的，已添加

WuHaobo · 2020-08-17T04:25:44Z

python/paddle/nn/layer/__init__.py

+from .common import ZeroPad2d  #DEFINE_ALIAS
+from .common import ReplicationPad3d  #DEFINE_ALIAS
+from .common import ConstantPad3d  #DEFINE_ALIAS
+from .common import CosineSimilarity


#DEFINE_ALIAS

WuHaobo · 2020-08-17T04:37:59Z

python/paddle/nn/layer/common.py

+            #    [ 5.  4.  5.  6.]]]]
+    """
+
+    def __init__(self, padding=[0, 0, 0, 0], data_format="NCHW", name=None):


这里padding应该咩有默认值吧；

已删除，多谢

willthefrog

LGTM

… add_pad

littletomatodonkey added 2 commits August 11, 2020 06:04

add pad func

671d2ff

add pad

53a713f

littletomatodonkey force-pushed the add_pad branch from 8e1ca75 to 53a713f Compare August 11, 2020 06:05

test=develop, add pad op and apis

2c09d75

LDOUBLEV reviewed Aug 11, 2020

View reviewed changes

littletomatodonkey added 4 commits August 11, 2020 09:08

restore pad2d

ac412e7

test=develop, fix paddl declare

2d423cd

fix pad interface

55c1189

test=develop, fix pad

98d11de

raindrops2sea requested a review from heavengate August 12, 2020 01:44

willthefrog requested changes Aug 13, 2020

View reviewed changes

willthefrog reviewed Aug 13, 2020

View reviewed changes

dyning reviewed Aug 13, 2020

View reviewed changes

test=develop, add all pad api and cos_sim

645fe90

willthefrog reviewed Aug 17, 2020

View reviewed changes

paddle/fluid/operators/pad3d_op.cc Show resolved Hide resolved

willthefrog reviewed Aug 17, 2020

View reviewed changes

WuHaobo reviewed Aug 17, 2020

View reviewed changes

littletomatodonkey added 4 commits August 17, 2020 05:37

test=develop, remove padding default value

85f975b

test=develop, rename var to tensor

b356382

test=develop, add more tests

9acc264

test=develop, fix conflict

10570b4

willthefrog previously approved these changes Aug 17, 2020

View reviewed changes

test=develop, rename tovar to totensor

a4abac9

littletomatodonkey dismissed willthefrog’s stale review via a4abac9 August 17, 2020 11:55

littletomatodonkey added 4 commits August 17, 2020 13:10

test=develop, fix init

835f82a

test=develop, add more test

fafd8b9

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

30ac1cc

… add_pad

test=develop, add more tests

ad220f7

raindrops2sea approved these changes Aug 19, 2020

View reviewed changes

willthefrog merged commit bcf0327 into PaddlePaddle:develop Aug 19, 2020

		@@ -254,91 +254,299 @@ def forward(self, input):
		return out


		class Pad2D(layers.Layer):

		self.assertRaises(Exception, test_reflect_2)

		self.assertRaises(Exception, test_reflect_3)

		self.assertTrue(np.allclose(output.numpy(), np_out))


		class TestPad1dClass(unittest.TestCase):

		self.assertTrue(np.allclose(output.numpy(), np_out))


		class TestPadAPI(unittest.TestCase):

add pad func #26106

add pad func #26106

Conversation

littletomatodonkey commented Aug 10, 2020

PR types

PR changes

Describe

paddle-bot-old bot commented Aug 10, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

willthefrog Aug 17, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

willthefrog Aug 17, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

willthefrog left a comment

Choose a reason for hiding this comment

willthefrog Aug 17, 2020 •

edited

Loading

willthefrog Aug 17, 2020 •

edited

Loading