修改COPY-FROM No.13 distributed #6004

jjyaoao · 2023-07-07T07:15:46Z

将代码块 .. code-block:: 使用 COPY-FROM 指令代替

PADDLEPADDLE_PR=55236

paddle-bot · 2023-07-07T07:15:51Z

感谢你贡献飞桨文档，文档预览构建中，Docs-New 跑完后即可预览，预览链接：http://preview-pr-6004.paddle-docs-preview.paddlepaddle.org.cn/documentation/docs/zh/api/index_cn.html
预览工具的更多说明，请参考：飞桨文档预览工具

sunzhongkai588

先改这些吧..感觉有挺多坑

docs/api/paddle/distributed/QueueDataset_cn.rst

sunzhongkai588 · 2023-07-11T11:47:26Z

docs/api/paddle/distributed/fleet/DistributedStrategy_cn.rst

-  strategy.recompute = True
-  strategy.recompute_configs = {"checkpoints": ["x"]}
-  strategy.save_to_prototxt("dist_strategy.prototxt")
+COPY-FROM: paddle.distributed.fleet.DistributedStrategy


Suggested change

COPY-FROM: paddle.distributed.fleet.DistributedStrategy

COPY-FROM: paddle.distributed.fleet.DistributedStrategy.save_to_prototxt

sunzhongkai588 · 2023-07-12T03:34:17Z

docs/api/paddle/distributed/fleet/Fleet_cn.rst

-
-    import paddle.distributed.fleet as fleet
-    fleet.init()
+COPY-FROM: paddle.distributed.fleet.Fleet:code-example1


这里是init方法下的代码示例，改成

Suggested change

COPY-FROM: paddle.distributed.fleet.Fleet:code-example1

COPY-FROM: paddle.distributed.fleet.Fleet.init:code-example1

另外，Fleet类的代码（paddle.distributed.fleet.Fleet:code-example1 和paddle.distributed.fleet.Fleet:code-example2）应该在方法之上进行引用

sunzhongkai588 · 2023-07-12T03:34:42Z

docs/api/paddle/distributed/fleet/Fleet_cn.rst

-
-    import paddle.distributed.fleet as fleet
-    fleet.init(is_collective=True)
+COPY-FROM: paddle.distributed.fleet.Fleet:code-example2


Suggested change

COPY-FROM: paddle.distributed.fleet.Fleet:code-example2

COPY-FROM: paddle.distributed.fleet.Fleet.init:code-example2

sunzhongkai588 · 2023-07-12T03:34:56Z

docs/api/paddle/distributed/fleet/Fleet_cn.rst

-    import paddle.distributed.fleet as fleet
-    role = fleet.PaddleCloudRoleMaker()
-    fleet.init(role)
+COPY-FROM: paddle.distributed.fleet.Fleet:code-example3


Suggested change

COPY-FROM: paddle.distributed.fleet.Fleet:code-example3

COPY-FROM: paddle.distributed.fleet.Fleet.init:code-example3

sunzhongkai588 · 2023-07-13T04:12:01Z

docs/api/paddle/distributed/fleet/Fleet_cn.rst

-
-    adam.step()
-    adam.clear_grad()
+COPY-FROM: paddle.distributed.fleet.Fleet.clear_grad


 minimize(loss, startup_program=None, parameter_list=None, no_grad_set=None)


minimize好像没有写内容，要不就把英文那部分内容翻译成中文搬过来吧，同时代码也copy from过来

docs/api/paddle/distributed/fleet/PaddleCloudRoleMaker_cn.rst

docs/api/paddle/distributed/fleet/UserDefinedRoleMaker_cn.rst

sunzhongkai588 · 2023-07-13T05:43:59Z

docs/api/paddle/distributed/fleet/UtilBase_cn.rst

-        # [8, 12]
-    if __name__ == "__main__":
-        train()
+COPY-FROM: paddle.distributed.fleet.UtilBase


Suggested change

COPY-FROM: paddle.distributed.fleet.UtilBase

COPY-FROM: paddle.distributed.fleet.UtilBase.all_reduce

docs/api/paddle/distributed/fleet/utils/HDFSClient_cn.rst

sunzhongkai588 · 2023-07-19T06:51:45Z

docs/api/paddle/distributed/fleet/utils/HDFSClient_cn.rst

+**代码示例**
+.. code-block:: text


Suggested change

**代码示例**

.. code-block:: text

**代码示例**

.. code-block:: text

docs/api/paddle/distributed/fleet/utils/HDFSClient_cn.rst

sunzhongkai588 · 2023-07-19T06:53:45Z

docs/api/paddle/distributed/fleet/utils/HDFSClient_cn.rst

所有的.. code-block:: text 上下都空一行吧，不然好像官网预览不出来，如：

docs/api/paddle/distributed/fleet/UserDefinedRoleMaker_cn.rst

docs/api/paddle/distributed/fleet/PaddleCloudRoleMaker_cn.rst

docs/api/paddle/distributed/QueueDataset_cn.rst

sunzhongkai588 · 2023-07-19T08:05:24Z

docs/api/paddle/distributed/fleet/DistributedStrategy_cn.rst

@SigureMo DistributedStrategy所有代码示例都没有成功过copyfrom，001师傅你看一下

奇怪，日志里明明显示已经找到了

sunzhongkai588 · 2023-07-19T08:06:15Z

docs/api/paddle/distributed/fleet/Fleet_cn.rst

@SigureMo Fleet有部分代码示例copy from不了，可能和tensor_cn.rst情况差不多，要不这个skip？

要不都 skip 吧，分布式太折磨了

要不都 skip 吧，分布式太折磨了

+1，这样这个任务可以早点结了

要不都 skip 吧，分布式太折磨了

其实可以把 paddle.distributed.fleet 下的都skip

和 @jjyaoao 讨论了下

DistributedStrategy_cn.rst 和 Fleet_cn.rst 就恢复原状（不改成copy from），到时候直接skip @SigureMo

PaddleCloudRoleMaker 和 UserDefinedRoleMaker 因为缺少英文文档，jj师傅已经写一半了，就连同copyfrom顺手改了吧

HDFSClient_cn.st 统一改成text，和英文对齐

Signed-off-by: jjyaoao <jjyaoao@126.com>

sunzhongkai588

PaddleCloudRoleMaker 和 UserDefinedRoleMaker 里的代码直接copy from 吧，因为我看paddle端都加上了。其他没有太大问题～

sunzhongkai588 · 2023-07-20T03:45:48Z

docs/api/paddle/distributed/fleet/PaddleCloudRoleMaker_cn.rst

-.. code-block:: python
+.. code-block:: text

-    import os
    import paddle.distributed.fleet as fleet
-
-    os.environ["PADDLE_PSERVER_NUMS"] = "2"
-    os.environ["PADDLE_TRAINERS_NUM"] = "2"
-
-    os.environ["POD_IP"] = "127.0.0.1"
-    os.environ["PADDLE_PORT"] = "36001"
-    os.environ["TRAINING_ROLE"] = "PSERVER"
-    os.environ["PADDLE_PSERVERS_IP_PORT_LIST"] = \
-        "127.0.0.1:36001,127.0.0.2:36001"
-
-    os.environ["PADDLE_TRAINER_ID"] = "0"
-
-    fleet.PaddleCloudRoleMaker(is_collective=False)
+    from paddle.distributed.fleet.base.role_maker import Role
+    fleet.UserDefinedRoleMaker(
+        current_id=0,
+        role=Role.SERVER,
+        worker_num=2,
+        server_endpoints=["127.0.0.1:36011", "127.0.0.1:36012"])


这块儿直接copy from PaddleCloudRoleMaker吧，我看 PaddlePaddle/Paddle#55236 英文源码那儿都加了

sunzhongkai588 · 2023-07-20T03:49:22Z

docs/api/paddle/distributed/fleet/UserDefinedRoleMaker_cn.rst

@@ -38,15 +37,13 @@ string



以上代码都直接copy from UserDefinedRoleMaker吧，因为英文源码都加上了

Signed-off-by: jjyaoao <jjyaoao@126.com>

sunzhongkai588

最后一个修改，queueDataset就不要copy from了，直接改成text..T.T

sunzhongkai588 · 2023-07-20T07:45:54Z

docs/api/paddle/distributed/QueueDataset_cn.rst

-
-    os.remove("./test_queue_dataset_run_a.txt")
-    os.remove("./test_queue_dataset_run_b.txt")
+COPY-FROM: paddle.distributed.QueueDataset.init


这个init方法还是别用copy from了...英文代码预览都不行，还是把block改成text吧

Signed-off-by: jjyaoao <jjyaoao@126.com>

sunzhongkai588

LGTM

paddle-bot bot added contributor status: proposed labels Jul 7, 2023

jjyaoao mentioned this pull request Jul 7, 2023

修改COPY-FROM No.13 distributed PaddlePaddle/Paddle#55236

Merged

luotao1 added the HappyOpenSource 快乐开源活动issue与PR label Jul 7, 2023

luotao1 assigned luotao1 and sunzhongkai588 Jul 7, 2023

luotao1 mentioned this pull request Jul 7, 2023

修改目前 Paddle docs 中仍使用 code-block 的示例代码为 COPY-FROM #5957

Closed

sunzhongkai588 reviewed Jul 13, 2023

View reviewed changes

jjyaoao force-pushed the jjy4 branch from 03b22fb to fb50385 Compare July 18, 2023 09:33

sunzhongkai588 reviewed Jul 19, 2023

View reviewed changes

jjyaoao force-pushed the jjy4 branch 2 times, most recently from 4ce6d38 to 6ca612c Compare July 19, 2023 09:21

修改COPY-FROM No.13 distributed

07365d6

Signed-off-by: jjyaoao <jjyaoao@126.com>

jjyaoao force-pushed the jjy4 branch from 6ca612c to 07365d6 Compare July 19, 2023 09:24

sunzhongkai588 reviewed Jul 20, 2023

View reviewed changes

update PaddleCloudRoleMaker and UserDefinedRoleMaker

12c6730

Signed-off-by: jjyaoao <jjyaoao@126.com>

sunzhongkai588 reviewed Jul 20, 2023

View reviewed changes

update QueueDataset_cn.rst

64592c6

Signed-off-by: jjyaoao <jjyaoao@126.com>

sunzhongkai588 approved these changes Jul 20, 2023

View reviewed changes

luotao1 merged commit 4eea9b9 into PaddlePaddle:develop Jul 21, 2023
3 checks passed

sunzhongkai588 mentioned this pull request Jul 24, 2023

[xdoctest] 分批次修改已有代码的示例 PaddlePaddle/Paddle#55629

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

修改COPY-FROM No.13 distributed #6004

修改COPY-FROM No.13 distributed #6004

jjyaoao commented Jul 7, 2023 •

edited

paddle-bot bot commented Jul 7, 2023

sunzhongkai588 left a comment

sunzhongkai588 Jul 11, 2023

sunzhongkai588 Jul 12, 2023

sunzhongkai588 Jul 12, 2023

sunzhongkai588 Jul 12, 2023

sunzhongkai588 Jul 13, 2023

sunzhongkai588 Jul 13, 2023

sunzhongkai588 Jul 19, 2023

sunzhongkai588 Jul 19, 2023

sunzhongkai588 Jul 19, 2023

SigureMo Jul 19, 2023

sunzhongkai588 Jul 19, 2023

SigureMo Jul 19, 2023

luotao1 Jul 19, 2023

sunzhongkai588 Jul 19, 2023

sunzhongkai588 Jul 19, 2023

sunzhongkai588 left a comment

sunzhongkai588 Jul 20, 2023

sunzhongkai588 Jul 20, 2023

sunzhongkai588 left a comment

sunzhongkai588 Jul 20, 2023

sunzhongkai588 left a comment

	COPY-FROM: paddle.distributed.fleet.DistributedStrategy
	COPY-FROM: paddle.distributed.fleet.DistributedStrategy.save_to_prototxt

	COPY-FROM: paddle.distributed.fleet.Fleet:code-example1
	COPY-FROM: paddle.distributed.fleet.Fleet.init:code-example1

	COPY-FROM: paddle.distributed.fleet.UtilBase
	COPY-FROM: paddle.distributed.fleet.UtilBase.all_reduce

修改COPY-FROM No.13 distributed #6004

修改COPY-FROM No.13 distributed #6004

Conversation

jjyaoao commented Jul 7, 2023 • edited

paddle-bot bot commented Jul 7, 2023

sunzhongkai588 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sunzhongkai588 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sunzhongkai588 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sunzhongkai588 left a comment

Choose a reason for hiding this comment

jjyaoao commented Jul 7, 2023 •

edited