You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The batch normalization followed the convolution or fully connected layer can be integrated with them. Doing so will give us a forward acceleration(about 30% in mobilenet) during inference.
Implementation
There are two simple examples:
conv without bias: conv->batch_norm->any_other_op should be changed to conv->elementwise_add (bias)->any_other_op
conv with bias: conv->elementwise_add (bias)->batch_norm->any_other_op should be changed to conv->elementwise_add (bias)->any_other_op
Thus, there are three~four stages when fusing batch normalization:
insert elementwise_add op, its input is the output of conv (this stage is only for conv without bias)
fuse the batch_norm's parameters to conv and elementwise_add
remove batch_norm ops and its variables which not used in any other ops.
adjust the input of any_other_op to be the output of elementwise_add, and remove unused variables again.
Motivation
The batch normalization followed the convolution or fully connected layer can be integrated with them. Doing so will give us a forward acceleration(about 30% in mobilenet) during inference.
Implementation
There are two simple examples:
conv->batch_norm->any_other_op
should be changed toconv->elementwise_add (bias)->any_other_op
conv->elementwise_add (bias)->batch_norm->any_other_op
should be changed toconv->elementwise_add (bias)->any_other_op
Thus, there are three~four stages when fusing batch normalization:
elementwise_add
op, its input is the output ofconv
(this stage is only for conv without bias)batch_norm
's parameters toconv
andelementwise_add
batch_norm
ops and its variables which not used in any other ops.any_other_op
to be the output ofelementwise_add
, and remove unused variables again.V2 implementation
#6704 by @NHZlX and a demo in mobile repo.
fluid implementation
We plan to use an inference transpiler to implement fuse batch normalization. Before this transpiler, we should implement:
insert_op
method for stage 1: refine sync_with_cpp for insert_op #9747remove_op
method for stage 3: remove vars when remove ops #9384 refine sync_with_cpp when remove ops or remove vars #9600 add remove_op, remove_var in Python end #9816remove_var
method for stage 4: add remove_var from c++ end #9607The text was updated successfully, but these errors were encountered: