Need to canonicalize or optimize high-dim concat to concat+transpose

[nadavrot]

The picture below depicts a concat node that joins nodes on dimension number 1. The IR that we generate for this code is inefficient for two reasons. First, we can't optimize the operator that writes the result because the result is scattered across the 2st dimension (dim zero is the first). And second, we emit a sequence of insert_tensor instructions that process the tensor several times invalidating cache. A much better way would be to represent this as dim-0 concat followed by a transpose.

Design question: I am not sure if this should be the canonical representation, the only representation or simply a target specific optimization.

[sparkingdark]

hey @nadavrot can you elaborate

[glowbucky]

Hey @nadavrot I would love to work on this issue. Can you please tell me where should I start, should this be designed as an graph optimization pass?