[AutoParallel] PHI copy support auto parallel #58436
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
你的PR提交成功，感谢你对开源项目的贡献! |
chenwhql
reviewed
Oct 27, 2023
test/auto_parallel/CMakeLists.txt
Outdated
| @@ -130,6 +130,11 @@ if(WITH_DISTRIBUTE AND WITH_GPU) | |||
| py_test_modules(test_gpt_with_newir MODULES test_gpt_with_newir) | |||
| set_tests_properties(test_gpt_with_newir | |||
| PROPERTIES LABELS "RUN_TYPE=EXCLUSIVE" TIMEOUT 100) | |||
| py_test_modules(test_semi_auto_parallel_functional_in_single_card MODULES | |||
There was a problem hiding this comment.
如果是单卡的话，可以看下注释，它这个cmake了做了分区，单卡的测试在最后
| def test_tensor_copy_to(self): | ||
| mesh = dist.ProcessMesh([0, 1], dim_names=["x"]) | ||
| dense_tensor = paddle.randn([10, 20]) | ||
| dist_tensor = dist.shard_tensor( |
There was a problem hiding this comment.
涉及切分的话，就有通信，单卡应该跑不过才对，或者用[None, None]不切分
paddle/phi/api/lib/tensor_copy.cc
Outdated
There was a problem hiding this comment.
phi/api下面的文件，在WITH_DISTRIBUTED=OFF的cmake参数下，需要用ifdef PADDLE_WITH_DISTRIBUTE宏，条件编译DistTensor相关分支
ForFishes
reviewed
Oct 30, 2023
| dense_tensor, | ||
| dist_attr=dist.DistAttr(mesh=mesh, sharding_specs=[None, None]), | ||
| ) | ||
| dist_tensor._copy_to(paddle.CPUPlace(), True) |
There was a problem hiding this comment.
copy的语义需要确定：1、mesh是否和设备相关，转cpu后，动半还缺少mesh和device的映射关系。2、如果是不同的distattr Dtensor，需要reshard 实现copy转换。
zeroRains
pushed a commit
to zeroRains/Paddle
that referenced
this pull request
Nov 8, 2023
* PHI copy support auto parallel
danleifeng
pushed a commit
to danleifeng/Paddle
that referenced
this pull request
Nov 14, 2023
* PHI copy support auto parallel
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
Others
PR changes
Others
Description
PHI API copy_to 支持 AutoParallel
Pcard-73145