Implement function inlining with the IR

[justinchuby]

Is this broadly also pattern replacement?

[titaiwangms]

cc @gramalingam

[titaiwangms]

Maybe this should be in ONNX repo?

[titaiwangms]

But I guess here we refer to inline_simple_function, which the function has less than/equal to 2 nodes

[gramalingam]

I would suggest that we understand the use-case before doing this. In particular:

• What do we want to do in the optimizer? We have both function-inlining as well as function-specialization currently. (By function specialization, I mean creation of a specialized graph for a particular call-site, but still retaining it as a function.)
• Benchmark evaluation with function-inlining.
• Investigate alternate implementation of function-rewrites (to see if we can get away with function inlining, without need for function specialization).

Until we understand the above better, we can use the ONNX function-inliner (where possible).

[justinchuby]

#1699

[justinchuby]

What do we want to do in the optimizer? We have both function-inlining as well as function-specialization currently. (By function specialization, I mean creation of a specialized graph for a particular call-site, but still retaining it as a function.)

I think if we can simplify a function we should, but specialization doesn't seem very useful so far.

the use-case before doing this

The motivation is to allow the optimize call to take an ir.Model. If we decide function inlining shouldn't happen in the optimize call that's ok too imo