Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add LayerNorm pass #761

Merged
merged 2 commits into from Mar 28, 2022
Merged

Conversation

wjj19950828
Copy link
Collaborator

@wjj19950828 wjj19950828 commented Mar 20, 2022

Create A Good Pull Request

下面的文字请保留在PR说明的最后面,并在提完PR后,根据实际情况勾选确认以下情况

Please check the follow step before merging this pull request

  • Python code style verification
  • Review all the code diff by yourself
  • All models(TensorFLow/Caffe/ONNX/PyTorch) testing passed
  • Details about your pull request, releated issues

If this PR add new model support, please update model_zoo.md and add model to out test model zoos(@wjj19950828)

  • New Model Supported
  • No New Model Supported

Do follow contributes

1、添加LayerNorm Pass,将onnx关于layernorm的碎片化op转为layernorm大op
优化前pd_model:
257e4f4160ce0fc0df82de203b837fe1
优化后pd_model:
c377178d07e442a6092a0fad42370601
2、VIT总体op数:优化前563个,优化后353个,op数减少近40%

@jiangjiajun jiangjiajun merged commit f44f507 into PaddlePaddle:develop Mar 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants