-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
se_resnet_152 multi-gpu profile #8661
Comments
#8624
|
@jacquesqiao could you post the link to the profiling script? |
profile model
https://github.com/PaddlePaddle/models/blob/develop/fluid/image_classification/se_resnext.py
profile with timeline tool
single card

two cards with nccl
two cards without nccl
some simple conclusion:
1, there is a huge gap between forward and backward when nccl is used
2, there is a huge gap between each step.
The text was updated successfully, but these errors were encountered: