Open
Description
I've been using ONNXScript extensively lately and really appreciate how powerful and flexible it is. As part of that experience, I and @Johansmm would like to suggest a few optimization patterns that could improve performance and simplify common model graphs. These are generally applicable across many models:
- Fold
BatchNormalization
into preceding nodes (Conv
,ConvTranspose
,Gemm
) Rewriter: Fold Batchnorm nodes #2312 - Fuse
MatMul + Add
intoGemm
- Fuse
Min + Max
intoClip
- Eliminate redundant
Reshape
/Flatten
nodes - Fold
Pad
intoConv
/ConvTranspose
- Fuse successive
Clip
/ReLU
nodes
We're happy to contribute if any of these are a good fit. Feel free to add other suggestions or let us know if some of these are out of scope.