Skip to content

[BYOC] [DNNL] enable in-place post-op sum in dnnl json runtime#12371

Merged
masahi merged 2 commits intoapache:mainfrom
yangulei:upstream_dnnl_inplace
Aug 12, 2022
Merged

[BYOC] [DNNL] enable in-place post-op sum in dnnl json runtime#12371
masahi merged 2 commits intoapache:mainfrom
yangulei:upstream_dnnl_inplace

Conversation

@yangulei
Copy link
Contributor

@yangulei yangulei commented Aug 11, 2022

This PR enable in-place post-op sum in dnnl json runtime for performance concern. This feature only enabled if the rhs of sum is not input nor output of the current dnnl partition. Thus the workloads with large dnnl partition will benefit most.
The correctness of this patch has been checked by the following 83 models from gluoncv:
['resnet18_v1', 'resnet34_v1', 'resnet50_v1', 'resnet101_v1', 'resnet152_v1', 'resnet18_v2', 'resnet34_v2', 'resnet50_v2', 'resnet101_v2', 'resnet152_v2', 'resnest14', 'resnest26', 'resnest50', 'resnest101', 'resnest200', 'resnest269', 'vgg11', 'vgg13', 'vgg16', 'vgg19', 'vgg11_bn', 'vgg13_bn', 'vgg16_bn', 'vgg19_bn', 'alexnet', 'densenet121', 'densenet161', 'densenet169', 'densenet201', 'squeezenet1.0', 'squeezenet1.1', 'googlenet', 'inceptionv3', 'xception', 'mobilenet1.0', 'mobilenet0.75', 'mobilenet0.5', 'mobilenet0.25', 'mobilenetv2_1.0', 'mobilenetv2_0.75', 'mobilenetv2_0.5', 'mobilenetv2_0.25', 'mobilenetv3_large', 'mobilenetv3_small', 'cifar_resnet20_v1', 'cifar_resnet56_v1', 'cifar_resnet110_v1', 'cifar_resnet20_v2', 'cifar_resnet56_v2', 'cifar_resnet110_v2', 'cifar_wideresnet16_10', 'cifar_wideresnet28_10', 'cifar_wideresnet40_8', 'cifar_resnext29_16x64d', 'resnet18_v1b', 'resnet34_v1b', 'resnet50_v1b', 'resnet101_v1b', 'resnet152_v1b', 'resnet50_v1c', 'resnet101_v1c', 'resnet152_v1c', 'resnet50_v1d', 'resnet101_v1d', 'resnet152_v1d', 'resnet50_v1s', 'resnet101_v1s', 'resnet152_v1s', 'resnext50_32x4d', 'resnext101_32x4d', 'resnext101_64x4d', 'se_resnext50_32x4d', 'se_resnext101_32x4d', 'se_resnext101_64x4d', 'senet_154', 'darknet53', 'resnet18_v1b_0.89', 'resnet50_v1d_0.86', 'resnet50_v1d_0.48', 'resnet50_v1d_0.37', 'resnet50_v1d_0.11', 'resnet101_v1d_0.76', 'resnet101_v1d_0.73']

@yangulei
Copy link
Contributor Author

@apeskov This is more like a work-around before the true solution based on memory planning is ready.

@masahi masahi merged commit e8de88e into apache:main Aug 12, 2022
@yangulei yangulei deleted the upstream_dnnl_inplace branch August 12, 2022 06:24
xinetzone pushed a commit to daobook/tvm that referenced this pull request Nov 25, 2022
…e#12371)

* enable inplace post-op sum in dnnl byoc

* add inplace post-op sum test
mikeseven pushed a commit to mikeseven/tvm that referenced this pull request Sep 27, 2023
…e#12371)

* enable inplace post-op sum in dnnl byoc

* add inplace post-op sum test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants