Essentially, we are upstreaming https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/python/tools/transformers/fusion_constant_fold.py
https://github.com/microsoft/onnxruntime/blob/838b97e73289cd11caf969f9f5c01ce153d6069f/onnxruntime/python/tools/transformers/dynamo_onnx_helper.py#L182
If initializer is not consumed by other inputs, we can transpose the initializer in advance.