onnxruntime_tools optimize a transformer model

[Chen1399]

Not a bug.
I have model like bert which have same moudle(embed/MultiHeadedAttention). But the forward process and input/output have a little difference. How can I optimize my model by onnxruntime_tools.
This is my try.

~/onnxruntime-master/onnxruntime/python/tools/transformers$ python3 optimizer.py --input model/encoder.onnx --output encoder_optim.onnx --model_type bert --num_heads 4 --hidden_size 320
               apply: Fused LayerNormalization count: 12
               apply: Fused SkipLayerNormalization count: 12
         prune_graph: Graph pruned: 0 inputs, 0 outputs and 0 nodes are removed
               apply: Fused SkipLayerNormalization(add bias) count: 12
            optimize: opset verion: 11
  save_model_to_file: Output model to encoder_optim.onnx
get_fused_operator_statistics: Optimized operators:{'EmbedLayerNormalization': 0, 'Attention': 0, 'Gelu': 0, 'FastGelu': 0, 'BiasGelu': 0, 'LayerNormalization': 0, 'SkipLayerNormalization': 12}
                main: The model has been optimized.

[tianleiwu]

@Chen1399, it seems that your model uses different activation function other than GELU, and only layer normalizations are optimized.

Could you share your onnx model and describe your model (like what is the different from huggingface bert model)?

The optimizer will match subgraph of ONNX model and fuse matched subgraph to operators like Attention etc. If your subgraph (like activation) is different, it might need some change to the optimizer to get all optimizations.

[stale]

This issue has been automatically marked as stale due to inactivity and will be closed in 7 days if no further activity occurs. If further support is needed, please provide an update and/or more details.

[stale]

This issue has been automatically closed due to inactivity. Please reactivate if further support is needed.