Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor hierarchical parallel cast autograd #5764

Merged
merged 188 commits into from
Aug 9, 2021
Merged

Refactor hierarchical parallel cast autograd #5764

merged 188 commits into from
Aug 9, 2021

Conversation

clackhan
Copy link
Contributor

@clackhan clackhan commented Aug 6, 2021

hierarchical_parallel_cast_op autograd 过程采用inplace操作

@clackhan
support_tensor_to/to_local
00cda8d
@clackhan
export consistent_tensor.to_local()
3a38a74
@clackhan
refine code
3053004
@clackhan
export tensor.to()...
65ad1c5
@clackhan
refine code
76fcd6f
@clackhan
refine code
ab4b2f5
@clackhan
Merge branch 'master' of https://github.com/Oneflow-Inc/oneflow into …
daf5bcc 
…support_tensor_to/to_local

Conflicts:
	oneflow/core/framework/tensor.h
@clackhan
optimize code
497fc82
@clackhan
refine code
e129aa2
@clackhan
Merge branch 'master' of https://github.com/Oneflow-Inc/oneflow into …
d5b7b39 
…support_tensor_to/to_local
@clackhan
Merge branch 'master' of https://github.com/Oneflow-Inc/oneflow into …
870909c 
…support_tensor_to/to_local

Conflicts:
	oneflow/core/framework/op_expr.cpp
	oneflow/core/framework/tensor.h
@clackhan
Merge branch 'master' into support_tensor_to/to_local
c572b50
@clackhan
Merge branch 'master' of https://github.com/Oneflow-Inc/oneflow into …
6546627 
…support_tensor_to/to_local

Conflicts:
	oneflow/core/framework/op_interpreter.h
@clackhan
Merge branch 'master' of https://github.com/Oneflow-Inc/oneflow into …
f7da2fe 
…support_tensor_to/to_local
@clackhan
Merge branch 'master' of https://github.com/Oneflow-Inc/oneflow into …
79ecc37 
…support_tensor_to/to_local

Conflicts:
	oneflow/api/python/symbol/sbp_symbol.cpp
@clackhan
refine
ca9e32a
@clackhan
Merge branch 'master' of https://github.com/Oneflow-Inc/oneflow into …
708732e 
…support_tensor_to/to_local
@clackhan
Merge branch 'master' of https://github.com/Oneflow-Inc/oneflow into …
f616d8a 
…support_tensor_to/to_local
@clackhan
Merge branch 'master' of https://github.com/Oneflow-Inc/oneflow into …
0bc6143 
…support_tensor_to/to_local

Conflicts:
	oneflow/api/python/framework/op_expr.cpp
	oneflow/api/python/framework/tensor.cpp
	oneflow/core/framework/op_expr.h
@clackhan
Merge branch 'master' of https://github.com/Oneflow-Inc/oneflow into …
b76498d 
…support_tensor_to/to_local
@clackhan
Merge branch 'master' of https://github.com/Oneflow-Inc/oneflow into …
aadf35d 
…support_tensor_to/to_local
@clackhan
back up
4b98830
@clackhan
Merge branch 'master' of https://github.com/Oneflow-Inc/oneflow into …
62b03e1 
…support_tensor_to/to_local
@clackhan
Merge branch 'master' of https://github.com/Oneflow-Inc/oneflow into …
75a3c20 
…support_tensor_to/to_local
@clackhan
add tensor.to func
c5c4cfb
@clackhan
make of_format
0aa5eb2
@clackhan
remove to in pyTensor
50d667c
@clackhan
Merge branch 'master' of https://github.com/Oneflow-Inc/oneflow into …
5fe6f3d 
…support_tensor_to/to_local

Conflicts:
	oneflow/api/python/framework/tensor.cpp
@clackhan
sync gpu data
4eaad64
@clackhan
Merge branch 'master' of https://github.com/Oneflow-Inc/oneflow into …
25f471c 
…support_tensor_to/to_local
lixinqi and others added 16 commits August 4, 2021 23:46
@lixinqi
Merge branch 'master' into dev_thread_local
b4f2e2a
@lixinqi
Merge branch 'master' into inferface_eager_boxing
1713683
@lixinqi
Merge branch 'dev_thread_local' into inferface_eager_boxing
3ba9155
@lixinqi
Delete thread_local_cache.h
ef7fbef
@lixinqi
Merge branch 'dev_thread_local' of github.com:Oneflow-Inc/oneflow int…
ab450a5 
…o inferface_eager_boxing
@lixinqi
bugfix: DeviceId4ParallelId -> MachineId4ParallelId
ad5c97c
@clackhan
optimize
6216608
@clackhan
Merge branch 'inferface_eager_boxing' of https://github.com/Oneflow-I…
36d0548 
…nc/oneflow into inferface_eager_boxing
@clackhan
Merge branch 'master' of https://github.com/Oneflow-Inc/oneflow into …
36de59a 
…inferface_eager_boxing

Conflicts:
	oneflow/core/functional/impl/consistent_cast.cpp
@clackhan
use inplace in grad of hierarchical_parallel_cas
25c2b92
@oneflow-ci-bot
Merge branch 'master' into inferface_eager_boxing
d4fd982
@oneflow-ci-bot
Merge branch 'master' into inferface_eager_boxing
68a7abd
@clackhan
Merge branch 'master' into inferface_eager_boxing
6100351
@oneflow-ci-bot
Merge branch 'master' into inferface_eager_boxing
2c3eca4
@clackhan
Merge branch 'master' of https://github.com/Oneflow-Inc/oneflow into …
33980ed 
…refactor_hierarchical_parallel_cast_autograd
@clackhan
Merge branch 'inferface_eager_boxing' of https://github.com/Oneflow-I…
2f3017a 
…nc/oneflow into refactor_hierarchical_parallel_cast_autograd
@clackhan clackhan requested a review from wyg1997 August 6, 2021 02:29
Base automatically changed from inferface_eager_boxing to master August 6, 2021 10:41
@clackhan
Merge branch 'master' of https://github.com/Oneflow-Inc/oneflow into …
162525d 
…refactor_hierarchical_parallel_cast_autograd

Conflicts:
	oneflow/core/autograd/gradient_funcs/hierarchical_parallel_cast.cpp
	oneflow/user/ops/eager_nccl_ops.cpp
@oneflow-ci-bot oneflow-ci-bot self-requested a review August 9, 2021 05:00
@github-actions
Copy link
Contributor

github-actions bot commented Aug 9, 2021

Speed stats: 
GPU Name: GeForce GTX 1080 

PyTorch resnet50 time: 136.9ms (= 6844.4ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 125.9ms (= 6294.0ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
Relative speed: 1.09 (= 136.9ms / 125.9ms)

PyTorch resnet50 time: 82.6ms (= 4130.2ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 72.7ms (= 3634.0ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
Relative speed: 1.14 (= 82.6ms / 72.7ms)

PyTorch resnet50 time: 55.7ms (= 2784.5ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 47.8ms (= 2391.5ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
Relative speed: 1.16 (= 55.7ms / 47.8ms)

PyTorch resnet50 time: 46.4ms (= 2321.2ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 50.7ms (= 2537.4ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
Relative speed: 0.91 (= 46.4ms / 50.7ms)

PyTorch resnet50 time: 40.6ms (= 2031.6ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 35.4ms (= 1767.6ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
Relative speed: 1.15 (= 40.6ms / 35.4ms)

@oneflow-ci-bot oneflow-ci-bot merged commit 1ca3b76 into master Aug 9, 2021
@oneflow-ci-bot oneflow-ci-bot deleted the refactor_hierarchical_parallel_cast_autograd branch August 9, 2021 06:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wyg1997 wyg1997 wyg1997 approved these changes

@lixinqi lixinqi Awaiting requested review from lixinqi

@oneflow-ci-bot oneflow-ci-bot Awaiting requested review from oneflow-ci-bot

Assignees
No one assigned
Labels
automerge enhancement system
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
6 participants
@clackhan @lixinqi @wyg1997 @oneflow-ci-bot @jackalcooper @hjchen2