Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fuse batch normalization #9629

Closed
luotao1 opened this issue Apr 4, 2018 · 0 comments
Closed

fuse batch normalization #9629

luotao1 opened this issue Apr 4, 2018 · 0 comments
Assignees
Labels
预测 原名Inference,包含Capi预测问题等

Comments

@luotao1
Copy link
Contributor

luotao1 commented Apr 4, 2018

Motivation

The batch normalization followed the convolution or fully connected layer can be integrated with them. Doing so will give us a forward acceleration(about 30% in mobilenet) during inference.

Implementation

There are two simple examples:

  • conv without bias: conv->batch_norm->any_other_op should be changed to conv->elementwise_add (bias)->any_other_op
  • conv with bias: conv->elementwise_add (bias)->batch_norm->any_other_op should be changed to conv->elementwise_add (bias)->any_other_op

Thus, there are three~four stages when fusing batch normalization:

  1. insert elementwise_add op, its input is the output of conv (this stage is only for conv without bias)
  2. fuse the batch_norm's parameters to conv and elementwise_add
  3. remove batch_norm ops and its variables which not used in any other ops.
  4. adjust the input of any_other_op to be the output of elementwise_add, and remove unused variables again.

V2 implementation

#6704 by @NHZlX and a demo in mobile repo.

fluid implementation

We plan to use an inference transpiler to implement fuse batch normalization. Before this transpiler, we should implement:

@luotao1 luotao1 self-assigned this Apr 4, 2018
@luotao1 luotao1 added the 预测 原名Inference,包含Capi预测问题等 label Apr 4, 2018
@Xreki Xreki added this to Basic Usage (DOING) in Inference Framework Apr 12, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
预测 原名Inference,包含Capi预测问题等
Projects
No open projects
Inference Framework
Basic Usage (DOING)
Development

No branches or pull requests

2 participants