Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dev reconstruct pad ops #6223

Merged
merged 18 commits into from
Sep 10, 2021
Merged

Dev reconstruct pad ops #6223

merged 18 commits into from
Sep 10, 2021

Conversation

Flowingsun007
Copy link
Contributor

@Flowingsun007 Flowingsun007 commented Sep 10, 2021

因为constant_pad1/2/3d op不支持在batch,channel维进行pad,而系统原有pad op同时支持1/2/3D pad且支持在N、C维pad,故此pr使用pad op替换了constant_pad1/2/3d op;

  • 使用原有的pad op替换constant_pad1/2/3d op
  • pad op的前向反向functor实现
  • pad op gradien funcs
  • 重构ConstantPad1/2/3d、ZeroPad2d module
  • 重构constant pad 1/2/3d auto testcase

重构后,在constant mode下,api对齐torch:https://pytorch.org/docs/stable/generated/torch.nn.functional.pad.html?highlight=pad#torch.nn.functional.pad

image

Copy link
Contributor

@hjchen2 hjchen2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

那之前你写的constant_pad还需要么,是不是可以删了

@Flowingsun007
Copy link
Contributor Author

Flowingsun007 commented Sep 10, 2021

那之前你写的constant_pad还需要么,是不是可以删了

嗯,是可以删了
(暂时先不删吧,我看single_client下也用到了这些op,后面在弄一个pr一起删吧)

@Flowingsun007 Flowingsun007 requested review from oneflow-ci-bot and removed request for oneflow-ci-bot September 10, 2021 02:51
@Flowingsun007 Flowingsun007 removed the request for review from oneflow-ci-bot September 10, 2021 03:26
@Flowingsun007 Flowingsun007 changed the title Dev reconstuct pad ops Dev reconstruct pad ops Sep 10, 2021
@github-actions
Copy link
Contributor

Speed stats:
GPU Name: GeForce GTX 1080 

OneFlow resnet50 time: 128.5ms (= 6426.6ms / 50, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 141.8ms (= 7089.1ms / 50, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.10 (= 141.8ms / 128.5ms)

OneFlow resnet50 time: 74.8ms (= 3738.3ms / 50, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 84.9ms (= 4246.6ms / 50, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.14 (= 84.9ms / 74.8ms)

OneFlow resnet50 time: 48.1ms (= 2404.2ms / 50, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 59.1ms (= 2956.9ms / 50, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.23 (= 59.1ms / 48.1ms)

OneFlow resnet50 time: 47.2ms (= 2362.1ms / 50, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 49.5ms (= 2473.0ms / 50, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.05 (= 49.5ms / 47.2ms)

OneFlow resnet50 time: 49.0ms (= 2447.8ms / 50, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 44.8ms (= 2238.9ms / 50, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 0.91 (= 44.8ms / 49.0ms)

OneFlow resnet50 time: 158.1ms (= 7903.3ms / 50, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 158.0ms (= 7900.3ms / 50, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.00 (= 158.0ms / 158.1ms)

OneFlow resnet50 time: 103.7ms (= 5184.4ms / 50, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 106.7ms (= 5337.5ms / 50, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.03 (= 106.7ms / 103.7ms)

OneFlow resnet50 time: 77.3ms (= 3867.2ms / 50, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 85.7ms (= 4285.3ms / 50, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.11 (= 85.7ms / 77.3ms)

OneFlow resnet50 time: 80.0ms (= 3998.6ms / 50, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 63.2ms (= 3158.8ms / 50, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 0.79 (= 63.2ms / 80.0ms)

OneFlow resnet50 time: 70.7ms (= 3534.1ms / 50, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 72.8ms (= 3638.9ms / 50, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.03 (= 72.8ms / 70.7ms)

@oneflow-ci-bot oneflow-ci-bot removed their request for review September 10, 2021 05:04
@oneflow-ci-bot oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot September 10, 2021 05:37
@github-actions
Copy link
Contributor

Speed stats:
GPU Name: GeForce GTX 1080 

OneFlow resnet50 time: 130.1ms (= 6504.0ms / 50, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 140.9ms (= 7047.0ms / 50, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.08 (= 140.9ms / 130.1ms)

OneFlow resnet50 time: 74.7ms (= 3733.2ms / 50, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 85.3ms (= 4265.2ms / 50, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.14 (= 85.3ms / 74.7ms)

OneFlow resnet50 time: 49.7ms (= 2485.5ms / 50, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 60.2ms (= 3010.3ms / 50, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.21 (= 60.2ms / 49.7ms)

OneFlow resnet50 time: 41.7ms (= 2082.8ms / 50, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 48.7ms (= 2432.7ms / 50, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.17 (= 48.7ms / 41.7ms)

OneFlow resnet50 time: 41.1ms (= 2054.6ms / 50, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 40.2ms (= 2011.4ms / 50, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 0.98 (= 40.2ms / 41.1ms)

OneFlow resnet50 time: 159.0ms (= 7949.5ms / 50, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 161.8ms (= 8091.7ms / 50, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.02 (= 161.8ms / 159.0ms)

OneFlow resnet50 time: 103.3ms (= 5166.4ms / 50, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 106.9ms (= 5346.3ms / 50, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.03 (= 106.9ms / 103.3ms)

OneFlow resnet50 time: 77.2ms (= 3859.2ms / 50, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 73.3ms (= 3663.5ms / 50, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 0.95 (= 73.3ms / 77.2ms)

OneFlow resnet50 time: 76.9ms (= 3845.9ms / 50, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 64.5ms (= 3224.0ms / 50, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 0.84 (= 64.5ms / 76.9ms)

OneFlow resnet50 time: 72.1ms (= 3605.3ms / 50, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 75.6ms (= 3779.6ms / 50, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.05 (= 75.6ms / 72.1ms)

@oneflow-ci-bot oneflow-ci-bot merged commit 6dc96c2 into master Sep 10, 2021
@oneflow-ci-bot oneflow-ci-bot deleted the dev_reconstuct_pad_ops branch September 10, 2021 07:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants