Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dev Align torch avgpool #5610

Merged
merged 35 commits into from
Aug 4, 2021
Merged

Dev Align torch avgpool #5610

merged 35 commits into from
Aug 4, 2021

Conversation

MARD1NO
Copy link
Contributor

@MARD1NO MARD1NO commented Jul 26, 2021

对齐Pytorch的avgpool系列

和maxpool部分参数不同,因此这个PR暂时还未考虑将这两个pool相关代码共用

doctest

image

@MARD1NO MARD1NO marked this pull request as ready for review July 27, 2021 08:17
@MARD1NO MARD1NO requested review from Flowingsun007 and Ldpe2G July 27, 2021 08:17
oneflow/core/functional/functional_api.yaml Outdated Show resolved Hide resolved
oneflow/user/kernels/avg_pooling_kernel.cu Outdated Show resolved Hide resolved
oneflow/user/kernels/avg_pooling_kernel.cu Show resolved Hide resolved
CHECK_EQ_OR_RETURN(stride.size(), dim);
for (int32_t stride_dim : stride) { CHECK_GT_OR_RETURN(stride_dim, 0); }
for (int32_t i = 0; i < padding.size(); i++) {
CHECK_GE_OR_RETURN(kernel_size[i], 2 * padding[i])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个限制为什么要存在呢

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image
pytorch有做对应的限制

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

好的

python/oneflow/nn/modules/pooling.py Outdated Show resolved Hide resolved
@@ -534,7 +313,7 @@ def __init__(
):
super().__init__()
self.kernel_size = _triple(kernel_size)
self.stride = _triple(stride) if stride is not None else _triple(kernel_size)
self.stride = _triple(stride) if (stride is not None) else _triple(kernel_size)
data_format = "NCDHW"
self.channel_pos = (
"channels_last" if data_format == "NDHWC" else "channels_first"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NDHW没听过,是不是写错了

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

D -> depth。也有人写成NTCHW,T->time

):
super().__init__()
self.kernel_size = _pair(kernel_size)
data_format = "NCHW" # only support "NCHW" for now !
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

如果是支持NCHW和NHWC那么可以删掉这个注释,否则加assert

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里暂时保留data_format参数,原因是AMP的dataformat会用到NHWC,如果有需要可以基于这个补充对应的NHWC版本kernel

python/oneflow/nn/modules/pooling.py Show resolved Hide resolved
@oneflow-ci-bot oneflow-ci-bot removed their request for review August 3, 2021 11:25
@oneflow-ci-bot oneflow-ci-bot self-requested a review August 3, 2021 15:01
@oneflow-ci-bot oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot August 3, 2021 16:04
@oneflow-ci-bot oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot August 3, 2021 17:16
@oneflow-ci-bot oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot August 3, 2021 18:37
@oneflow-ci-bot oneflow-ci-bot self-requested a review August 3, 2021 22:37
@oneflow-ci-bot oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot August 3, 2021 23:53
@oneflow-ci-bot oneflow-ci-bot self-requested a review August 4, 2021 01:55
@MARD1NO MARD1NO removed the request for review from oneflow-ci-bot August 4, 2021 02:49
@MARD1NO MARD1NO requested a review from oneflow-ci-bot August 4, 2021 02:55
@oneflow-ci-bot oneflow-ci-bot removed their request for review August 4, 2021 03:39
@oneflow-ci-bot oneflow-ci-bot self-requested a review August 4, 2021 03:39
@github-actions
Copy link
Contributor

github-actions bot commented Aug 4, 2021

Speed stats:
GPU Name: GeForce GTX 1080 

PyTorch resnet50 time: 139.6ms (= 6979.5ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 125.6ms (= 6278.7ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
Relative speed: 1.11 (= 139.6ms / 125.6ms)

PyTorch resnet50 time: 83.3ms (= 4165.4ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 72.8ms (= 3640.3ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
Relative speed: 1.14 (= 83.3ms / 72.8ms)

PyTorch resnet50 time: 57.3ms (= 2863.4ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 46.5ms (= 2326.3ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
Relative speed: 1.23 (= 57.3ms / 46.5ms)

PyTorch resnet50 time: 49.2ms (= 2457.6ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 43.1ms (= 2152.8ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
Relative speed: 1.14 (= 49.2ms / 43.1ms)

PyTorch resnet50 time: 43.5ms (= 2172.6ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 38.6ms (= 1931.2ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
Relative speed: 1.12 (= 43.5ms / 38.6ms)

@oneflow-ci-bot oneflow-ci-bot merged commit 1b8faab into master Aug 4, 2021
@oneflow-ci-bot oneflow-ci-bot deleted the align_torch_avgpool branch August 4, 2021 05:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants