-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Relay][transform][SimplifyExpr] simplify adjacent muls and adds with constants #13213
Conversation
Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment.
Generated by tvm-bot |
@vinx13 This PR could also solve the issue about |
tvm/tests/python/relay/test_pass_partition_graph.py Lines 1086 to 1089 in 5c9066d
@masahi The tests above failed because the pattern matcher always matches the last two add s in conv-add[bias]-add[bn]-add[sum] , but the last add don't have any constant inputs thus cannot be simplified. Do you have any idea to fix this?
In this PR, I generalize the |
This sounds tricky, since I'd expect the pattern matcher to be operating in a "bottom up" manner. Is it possible to add "constant-ness" condition in the pattern? Otherwise I can only think of adding conv2d into the |
The pattern with Take Is there a way to evaluate the |
Since we run tvm/src/relay/backend/utils.cc Lines 264 to 266 in f7f2cda
|
tvm/src/relay/transforms/simplify_expr.cc Lines 786 to 790 in de7f762
The iterations happen inside SimplifyExpr , we need to find a way to evaluate the const expr manually.
Yes, but only for the const expr already exists before |
@masahi Problem solved after applying |
… constants (apache#13213) * simplify adjacent muls and adds with constants * apply FoldConstant inside SimplifyExpr
… constants (apache#13213) * simplify adjacent muls and adds with constants * apply FoldConstant inside SimplifyExpr
This PR enables simplification and folding of a sub graph containing adjacent
mul
s andadd
s with constant inputs.Motivation
Workloads like densenet-121 has several partitions with
conv-bn-mul-add-relu
pattern, for example:Current transforms on this pattern are:
conv-bn-mul-add-relu
as the original pattern.conv-mul-add-mul-add-relu
asbn
is expended tomul-add
.conv-add-mul-add-relu
as the firstmul
is folded intoconv
.As all the
mul
s andadd
s have constant second inputs, they should be folded to a singlemul-add
and a preferred transform sequence should be:conv-bn-mul-add-relu
as the original pattern.conv-mul-add-mul-add-relu
asbn
is expended tomul-add
.conv-mul-add-relu
asmul
s andadd
s are folded to one singlemul-add
.conv-add-relu
asmul
is folded intoconv
.Solution
Actually, any series contain
mul
s andadd
s with constant inputs could be folded to one particularmul-add
. Three rewrite rules are added to make this happen:mul-mul
->mul
add-add
->add
add-mul
->mul-add
As
SimplifyExpr
apply simplifications iteratively until no changes to the graph, anymul
andadd
series could be rewritten to one singlemul
,add
ormul-add
with one of the binary inputs could evaluates to a constant in the followingFoldConstant
pass.