Update the demo code and the doc of varbase.backward. #26506

wzzju · 2020-08-20T13:17:19Z

PR types

Others

PR changes

Docs

Describe

Update the demo code and the doc of varbase.backward. The pr of FluidDoc about backward.

Variable.backward

paddle.grad

paddle-bot-old · 2020-08-20T13:17:26Z

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

zhhsplendid · 2020-08-21T10:13:39Z

paddle/fluid/pybind/imperative.cc

@@ -513,28 +513,30 @@ void BindImperative(py::module *m_ptr) {
    Attribute:
        **sort_sum_gradient**:

-        If framework will sum the gradient by the reverse order of trace. eg. x_var ( :ref:`api_guide_Variable` ) will be the input of multiple OP such as :ref:`api_fluid_layers_scale` , this attr will decide if framework will sum gradient of `x_var` by the reverse order.
+        If framework will sum gradients by the reverse order of the forward execution sequence. 


"If True, framework will ..."

Otherwise this sentence doesn't make sense.

zhhsplendid · 2020-08-21T10:34:15Z

paddle/fluid/pybind/imperative.cc

@@ -513,28 +513,30 @@ void BindImperative(py::module *m_ptr) {
    Attribute:
        **sort_sum_gradient**:

-        If framework will sum the gradient by the reverse order of trace. eg. x_var ( :ref:`api_guide_Variable` ) will be the input of multiple OP such as :ref:`api_fluid_layers_scale` , this attr will decide if framework will sum gradient of `x_var` by the reverse order.
+        If framework will sum gradients by the reverse order of the forward execution sequence. 
+        For instance, x_var ( :ref:`api_guide_Variable` ) will be the input of multiple OP such as :ref:`api_fluid_layers_scale` ,


"For instane, when/if x_var ( :ref:api_guide_Variable ) is the input of multiple OPs such as :ref:api_fluid_layers_scale, whether the framework sums the gradient of x_var by the reverse order is determined by this attr".

There are two grammar knowledges:

When we say conditional things, or things happen commonly, we don't use future tense. For example: "if x_var is XXX". no need to "if x_var will be XXX".

"this attr will decide XXX" is fine to me but I feel my writing is better.

Thanks. Done.

zhhsplendid · 2020-08-21T10:34:52Z

paddle/fluid/pybind/imperative.cc

+                # x_var will be multi-scales' input here
+                for _ in range(10):
+                    sums_inputs.append(paddle.scale(x_var))
+                ret2 = paddle.sums(sums_inputs)


Why is ret2, loss2 without ret1, loss1? Can we just write ret, loss?

zhhsplendid · 2020-08-21T10:36:18Z

python/paddle/fluid/dygraph/varbase_patch_methods.py

+            backward_strategy(BackwardStrategy, optional): The backward strategy to run the backward pass. If you would
+                like to sum gradients by the reverse order of the forward execution sequence, please set the value of
+                :code:`BackwardStrategy.sort_sum_gradient` to True. Refer to :ref:`api_fluid_dygraph_BackwardStrategy` for
+                more details. Defaults to None, which meant that the value of :code:`BackwardStrategy.sort_sum_gradient` is False.


"which meant" -> "which means"

zhhsplendid · 2020-08-21T10:37:17Z

python/paddle/fluid/dygraph/varbase_patch_methods.py

-                    backward_strategy = fluid.dygraph.BackwardStrategy()
-                    backward_strategy.sort_sum_gradient = True
-                    loss2.backward(backward_strategy)
+                inputs2 = []


Same to above comment, why trailing 2 at these variables.

zhhsplendid · 2020-08-21T10:38:06Z

python/paddle/fluid/framework.py

+            backward_strategy(BackwardStrategy, optional): The backward strategy to run the backward pass. If you would
+                like to sum gradients by the reverse order of the forward execution sequence, please set the value of
+                :code:`BackwardStrategy.sort_sum_gradient` to True. Refer to :ref:`api_fluid_dygraph_BackwardStrategy` for
+                more details. Defaults to None, which meant that the value of :code:`BackwardStrategy.sort_sum_gradient` is False.


"which meant" -> "which means"

zhhsplendid · 2020-08-21T10:38:25Z

python/paddle/fluid/framework.py

-                    backward_strategy = fluid.dygraph.BackwardStrategy()
-                    backward_strategy.sort_sum_gradient = True
-                    loss2.backward(backward_strategy)
+                inputs2 = []


Same to above comment, why trailing 2 at these variables.

swtkiwi · 2020-08-21T10:54:49Z

paddle/fluid/pybind/imperative.cc

-                    backward_strategy = fluid.dygraph.BackwardStrategy()
-                    backward_strategy.sort_sum_gradient = True
-                    loss2.backward(backward_strategy)
+                x_var = paddle.to_variable(x)


paddle.to_variable->paddle.to_tensor

swtkiwi · 2020-08-21T10:55:48Z

python/paddle/fluid/dygraph/varbase_patch_methods.py

@@ -134,38 +134,41 @@ def backward(self, backward_strategy=None, retain_graph=False):
        **Notes**:
            **This API is ONLY available in Dygraph mode**

-        Run backward of current Graph which starts from current Variable
+        Run backward of current Graph which starts from current Variable.



Variable->Tensor

swtkiwi · 2020-08-21T10:56:26Z

python/paddle/fluid/dygraph/varbase_patch_methods.py

-                    loss2.backward(backward_strategy)
+                inputs2 = []
+                for _ in range(10):
+                    tmp = paddle.to_variable(x)


paddle.to_variable->paddle.to_tensor

… fix_backward_doc

TCChenlong

LGTM

… fix_backward_doc

XiaoguangHu01

LGTM

TCChenlong

LGTM

raindrops2sea

LGTM

update the demo code and the doc of varbase.backward.

114071b

wzzju force-pushed the fix_backward_doc branch 2 times, most recently from 7a5e807 to 2990d29 Compare August 21, 2020 07:15

update the doc of the fake interface paddle.fluid.Variable.

06b996a

wzzju force-pushed the fix_backward_doc branch from 2990d29 to 06b996a Compare August 21, 2020 07:28

wzzju mentioned this pull request Aug 21, 2020

Update the demo code and the doc of varbase.backward. PaddlePaddle/docs#2448

Merged

zhhsplendid reviewed Aug 21, 2020

View reviewed changes

swtkiwi reviewed Aug 21, 2020

View reviewed changes

fix some grammar errors.

24d5230

wzzju force-pushed the fix_backward_doc branch from 821215c to 24d5230 Compare August 21, 2020 11:12

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

7e6e711

… fix_backward_doc

TCChenlong previously approved these changes Aug 21, 2020

View reviewed changes

wzzju added 2 commits August 26, 2020 17:14

remove BackwardStrategy.

9d42fc9

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

9cf12ba

… fix_backward_doc

wzzju dismissed TCChenlong’s stale review via 9cf12ba August 26, 2020 17:15

XiaoguangHu01 approved these changes Aug 27, 2020

View reviewed changes

phlrain self-requested a review August 27, 2020 03:44

TCChenlong approved these changes Aug 27, 2020

View reviewed changes

luotao1 approved these changes Aug 27, 2020

View reviewed changes

raindrops2sea approved these changes Aug 27, 2020

View reviewed changes

wzzju merged commit f9066e6 into PaddlePaddle:develop Aug 27, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update the demo code and the doc of varbase.backward. #26506

Update the demo code and the doc of varbase.backward. #26506

wzzju commented Aug 20, 2020 •

edited

paddle-bot-old bot commented Aug 20, 2020

zhhsplendid Aug 21, 2020

wzzju Aug 21, 2020

zhhsplendid Aug 21, 2020

wzzju Aug 21, 2020 •

edited

zhhsplendid Aug 21, 2020

wzzju Aug 21, 2020

zhhsplendid Aug 21, 2020

wzzju Aug 21, 2020

zhhsplendid Aug 21, 2020

wzzju Aug 21, 2020

zhhsplendid Aug 21, 2020

wzzju Aug 21, 2020

zhhsplendid Aug 21, 2020

wzzju Aug 21, 2020

swtkiwi Aug 21, 2020

wzzju Aug 21, 2020

swtkiwi Aug 21, 2020

wzzju Aug 21, 2020

swtkiwi Aug 21, 2020

wzzju Aug 21, 2020 •

edited

TCChenlong left a comment

XiaoguangHu01 left a comment

TCChenlong left a comment

raindrops2sea left a comment

Update the demo code and the doc of varbase.backward. #26506

Update the demo code and the doc of varbase.backward. #26506

Conversation

wzzju commented Aug 20, 2020 • edited

PR types

PR changes

Describe

Variable.backward

paddle.grad

paddle-bot-old bot commented Aug 20, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wzzju Aug 21, 2020 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wzzju Aug 21, 2020 • edited

Choose a reason for hiding this comment

TCChenlong left a comment

Choose a reason for hiding this comment

XiaoguangHu01 left a comment

Choose a reason for hiding this comment

TCChenlong left a comment

Choose a reason for hiding this comment

raindrops2sea left a comment

Choose a reason for hiding this comment

wzzju commented Aug 20, 2020 •

edited

wzzju Aug 21, 2020 •

edited

wzzju Aug 21, 2020 •

edited