Layout transform: Fix-up QDQ units and add constant folding

[adrianlizarraga]

Description

Problem 1: Broken Transpose QDQ unit

Layout transform's specialized cost function aggressively pushes down transposes with channel-first or channel-last perms. This can lead to a situation where a channel-fist/last Transpose gets stuck after being pushed through an Unsqueeze node that makes the Transpose's perm no longer channel-first/last. At this point, the specialized cost function defers to the default const function, which does not see a need to continue pushing this transpose node. This breaks the QDQ node units for both the Unsqueeze and the Transpose: DQ -> Unsqueeze -> Transpose -> Q.

The transpose optimizer should insert a Q -> DQ pair between the Unsqueeze and Transpose nodes to fix both QDQ node units: DQ -> Unsqueeze -> Q[new] -> DQ[new] -> Transpose -> Q

Problem 2: Inserted Squeeze/Transpose nodes should be constant folded when possible.

The transpose optimizer inserts Squeeze (and Transpose) ops between an initializer and a DQ to counteract the effect of Unsqueezing that initializer if it is consumed by multiple nodes. This results in a graph where the inserted nodes are not in valid node units:

Original graph where two Mul nodes share a common initializer input:

Resulting graph after transpose optimization without constant folding:

Here, the circled Transpose and Squeeze nodes operate on a quantized integer type but are not in valid QDQ node units. The solution is to run constant folding, which results in:

Motivation and Context

Improve the layout transformation to allow more models to run on EPs that prefer the channel-last layout.