Feature Request - Dealing with int64 in the exported ONNX model #233

snakers4 · 2020-10-07T07:24:50Z

Hi,

Background

We are trying to run models from silero-models via onnx.js.
We have had various issues with ONNX export, but most of them finally were resolved using built-in messages and just testing using onnxruntime (we had to heavily simplify some model parts).

Model Export

The models were ported as follows:

Original model in PyTorch =>
Fusing convolutions (w/o quantization) =>
Simplified model in TorchScript =>
ONNX (=> TensorFlow via onnx-tensorflow)

Model export script in PyTorch (step 4 above)

torch.onnx.export(onnx_model,                # model being run
                  inputs,                    # model input (or a tuple for multiple inputs)
                  "en_v1_test.onnx",         # where to save the model (can be a file or file-like object)
                  export_params=True,        # store the trained parameter weights inside the model file
                  opset_version=12,          # the ONNX version to export the model to
                  do_constant_folding=True,  # whether to execute constant folding for optimization
                  input_names = ['input'],   # the model's input names
                  output_names = ['output'], # the model's output names
                  dynamic_axes={'input' : {0: 'batch',
                                           1: 'samples'},    
                                'output' : {0: 'batch',
                                            1: 'frames'}},
                  verbose=True
                 )

We have tested that the converted models work fine with:

onnxruntime
TensorFlow (on CPU and GPU)

Problem

But when we try to run the model in onnx-js, we face an issue similar to #168 .

Upon closer inspection via netron app we see that:

slicing operators (static and dynamic alike) are stored as int64 values
some constants like 0, 1, -1 are stored as int64 values

If we then inspect PyTorch export log, we can see that it has Long() = 228 times, so I believe it is not some idiosyncrasy of our models (except for the first normalization and STFT we mostly use off-the-shelf components) but a feature of ONNX export in general.

Model conversion log (very verbose!)

graph(%input : Float(1:73728, 73728:1),
    %stft.forward_basis : Float(322:320, 1:0, 320:1),
    %audio_normalize.filter_ : Float(1:161, 1:161, 161:1),
    %encoder.0.layers.0.0.weight : Float(512:1127, 161:7, 7:1),
    %encoder.0.layers.0.0.bias : Float(512:1),
    %encoder.0.layers.4.0.weight : Float(512:512, 512:1, 1:1),
    %encoder.0.layers.4.0.bias : Float(512:1),
    %encoder.1.layers.0.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.1.layers.0.0.bias : Float(512:1),
    %encoder.1.layers.4._se_reduce.weight : Float(102:512, 512:1, 1:1),
    %encoder.1.layers.4._se_reduce.bias : Float(102:1),
    %encoder.1.layers.4._se_expand.weight : Float(512:102, 102:1, 1:1),
    %encoder.1.layers.4._se_expand.bias : Float(512:1),
    %encoder.1.layers.5.0.weight : Float(512:512, 512:1, 1:1),
    %encoder.1.layers.5.0.bias : Float(512:1),
    %encoder.1.layers.9.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.1.layers.9.0.bias : Float(512:1),
    %encoder.1.layers.13._se_reduce.weight : Float(102:512, 512:1, 1:1),
    %encoder.1.layers.13._se_reduce.bias : Float(102:1),
    %encoder.1.layers.13._se_expand.weight : Float(512:102, 102:1, 1:1),
    %encoder.1.layers.13._se_expand.bias : Float(512:1),
    %encoder.1.layers.14.0.weight : Float(512:512, 512:1, 1:1),
    %encoder.1.layers.14.0.bias : Float(512:1),
    %encoder.2.layers.0.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.2.layers.0.0.bias : Float(512:1),
    %encoder.2.layers.4._se_reduce.weight : Float(102:512, 512:1, 1:1),
    %encoder.2.layers.4._se_reduce.bias : Float(102:1),
    %encoder.2.layers.4._se_expand.weight : Float(512:102, 102:1, 1:1),
    %encoder.2.layers.4._se_expand.bias : Float(512:1),
    %encoder.2.layers.5.0.weight : Float(512:512, 512:1, 1:1),
    %encoder.2.layers.5.0.bias : Float(512:1),
    %encoder.2.layers.9.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.2.layers.9.0.bias : Float(512:1),
    %encoder.2.layers.13._se_reduce.weight : Float(102:512, 512:1, 1:1),
    %encoder.2.layers.13._se_reduce.bias : Float(102:1),
    %encoder.2.layers.13._se_expand.weight : Float(512:102, 102:1, 1:1),
    %encoder.2.layers.13._se_expand.bias : Float(512:1),
    %encoder.2.layers.14.0.weight : Float(512:512, 512:1, 1:1),
    %encoder.2.layers.14.0.bias : Float(512:1),
    %encoder.3.layers.0.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.3.layers.0.0.bias : Float(512:1),
    %encoder.3.layers.4._se_reduce.weight : Float(102:512, 512:1, 1:1),
    %encoder.3.layers.4._se_reduce.bias : Float(102:1),
    %encoder.3.layers.4._se_expand.weight : Float(512:102, 102:1, 1:1),
    %encoder.3.layers.4._se_expand.bias : Float(512:1),
    %encoder.3.layers.5.0.weight : Float(512:512, 512:1, 1:1),
    %encoder.3.layers.5.0.bias : Float(512:1),
    %encoder.3.layers.9.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.3.layers.9.0.bias : Float(512:1),
    %encoder.3.layers.13._se_reduce.weight : Float(102:512, 512:1, 1:1),
    %encoder.3.layers.13._se_reduce.bias : Float(102:1),
    %encoder.3.layers.13._se_expand.weight : Float(512:102, 102:1, 1:1),
    %encoder.3.layers.13._se_expand.bias : Float(512:1),
    %encoder.3.layers.14.0.weight : Float(512:512, 512:1, 1:1),
    %encoder.3.layers.14.0.bias : Float(512:1),
    %encoder.4.layers.0.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.4.layers.0.0.bias : Float(512:1),
    %encoder.4.layers.4._se_reduce.weight : Float(102:512, 512:1, 1:1),
    %encoder.4.layers.4._se_reduce.bias : Float(102:1),
    %encoder.4.layers.4._se_expand.weight : Float(512:102, 102:1, 1:1),
    %encoder.4.layers.4._se_expand.bias : Float(512:1),
    %encoder.4.layers.5.0.weight : Float(512:512, 512:1, 1:1),
    %encoder.4.layers.5.0.bias : Float(512:1),
    %encoder.4.layers.9.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.4.layers.9.0.bias : Float(512:1),
    %encoder.4.layers.13._se_reduce.weight : Float(102:512, 512:1, 1:1),
    %encoder.4.layers.13._se_reduce.bias : Float(102:1),
    %encoder.4.layers.13._se_expand.weight : Float(512:102, 102:1, 1:1),
    %encoder.4.layers.13._se_expand.bias : Float(512:1),
    %encoder.4.layers.14.0.weight : Float(512:512, 512:1, 1:1),
    %encoder.4.layers.14.0.bias : Float(512:1),
    %encoder.5.layers.0.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.5.layers.0.0.bias : Float(512:1),
    %encoder.5.layers.4._se_reduce.weight : Float(102:512, 512:1, 1:1),
    %encoder.5.layers.4._se_reduce.bias : Float(102:1),
    %encoder.5.layers.4._se_expand.weight : Float(512:102, 102:1, 1:1),
    %encoder.5.layers.4._se_expand.bias : Float(512:1),
    %encoder.5.layers.5.0.weight : Float(512:512, 512:1, 1:1),
    %encoder.5.layers.5.0.bias : Float(512:1),
    %encoder.5.layers.9.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.5.layers.9.0.bias : Float(512:1),
    %encoder.5.layers.13._se_reduce.weight : Float(102:512, 512:1, 1:1),
    %encoder.5.layers.13._se_reduce.bias : Float(102:1),
    %encoder.5.layers.13._se_expand.weight : Float(512:102, 102:1, 1:1),
    %encoder.5.layers.13._se_expand.bias : Float(512:1),
    %encoder.5.layers.14.0.weight : Float(512:512, 512:1, 1:1),
    %encoder.5.layers.14.0.bias : Float(512:1),
    %encoder.6.layers.0.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.6.layers.0.0.bias : Float(512:1),
    %encoder.6.layers.4._se_reduce.weight : Float(102:512, 512:1, 1:1),
    %encoder.6.layers.4._se_reduce.bias : Float(102:1),
    %encoder.6.layers.4._se_expand.weight : Float(512:102, 102:1, 1:1),
    %encoder.6.layers.4._se_expand.bias : Float(512:1),
    %encoder.6.layers.5.0.weight : Float(512:512, 512:1, 1:1),
    %encoder.6.layers.5.0.bias : Float(512:1),
    %encoder.6.layers.9.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.6.layers.9.0.bias : Float(512:1),
    %encoder.6.layers.13._se_reduce.weight : Float(102:512, 512:1, 1:1),
    %encoder.6.layers.13._se_reduce.bias : Float(102:1),
    %encoder.6.layers.13._se_expand.weight : Float(512:102, 102:1, 1:1),
    %encoder.6.layers.13._se_expand.bias : Float(512:1),
    %encoder.6.layers.14.0.weight : Float(512:512, 512:1, 1:1),
    %encoder.6.layers.14.0.bias : Float(512:1),
    %encoder.7.layers.0.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.7.layers.0.0.bias : Float(512:1),
    %encoder.7.layers.4.0.weight : Float(512:512, 512:1, 1:1),
    %encoder.7.layers.4.0.bias : Float(512:1),
    %encoder.8.layers.0.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.8.layers.0.0.bias : Float(512:1),
    %encoder.8.layers.4._se_reduce.weight : Float(102:512, 512:1, 1:1),
    %encoder.8.layers.4._se_reduce.bias : Float(102:1),
    %encoder.8.layers.4._se_expand.weight : Float(512:102, 102:1, 1:1),
    %encoder.8.layers.4._se_expand.bias : Float(512:1),
    %encoder.8.layers.5.0.weight : Float(512:512, 512:1, 1:1),
    %encoder.8.layers.5.0.bias : Float(512:1),
    %encoder.8.layers.9.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.8.layers.9.0.bias : Float(512:1),
    %encoder.8.layers.13._se_reduce.weight : Float(102:512, 512:1, 1:1),
    %encoder.8.layers.13._se_reduce.bias : Float(102:1),
    %encoder.8.layers.13._se_expand.weight : Float(512:102, 102:1, 1:1),
    %encoder.8.layers.13._se_expand.bias : Float(512:1),
    %encoder.8.layers.14.0.weight : Float(512:512, 512:1, 1:1),
    %encoder.8.layers.14.0.bias : Float(512:1),
    %encoder.9.layers.0.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.9.layers.0.0.bias : Float(512:1),
    %encoder.9.layers.4._se_reduce.weight : Float(102:512, 512:1, 1:1),
    %encoder.9.layers.4._se_reduce.bias : Float(102:1),
    %encoder.9.layers.4._se_expand.weight : Float(512:102, 102:1, 1:1),
    %encoder.9.layers.4._se_expand.bias : Float(512:1),
    %encoder.9.layers.5.0.weight : Float(512:512, 512:1, 1:1),
    %encoder.9.layers.5.0.bias : Float(512:1),
    %encoder.9.layers.9.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.9.layers.9.0.bias : Float(512:1),
    %encoder.9.layers.13._se_reduce.weight : Float(102:512, 512:1, 1:1),
    %encoder.9.layers.13._se_reduce.bias : Float(102:1),
    %encoder.9.layers.13._se_expand.weight : Float(512:102, 102:1, 1:1),
    %encoder.9.layers.13._se_expand.bias : Float(512:1),
    %encoder.9.layers.14.0.weight : Float(512:512, 512:1, 1:1),
    %encoder.9.layers.14.0.bias : Float(512:1),
    %encoder.10.layers.0.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.10.layers.0.0.bias : Float(512:1),
    %encoder.10.layers.4._se_reduce.weight : Float(102:512, 512:1, 1:1),
    %encoder.10.layers.4._se_reduce.bias : Float(102:1),
    %encoder.10.layers.4._se_expand.weight : Float(512:102, 102:1, 1:1),
    %encoder.10.layers.4._se_expand.bias : Float(512:1),
    %encoder.10.layers.5.0.weight : Float(512:512, 512:1, 1:1),
    %encoder.10.layers.5.0.bias : Float(512:1),
    %encoder.10.layers.9.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.10.layers.9.0.bias : Float(512:1),
    %encoder.10.layers.13._se_reduce.weight : Float(102:512, 512:1, 1:1),
    %encoder.10.layers.13._se_reduce.bias : Float(102:1),
    %encoder.10.layers.13._se_expand.weight : Float(512:102, 102:1, 1:1),
    %encoder.10.layers.13._se_expand.bias : Float(512:1),
    %encoder.10.layers.14.0.weight : Float(512:512, 512:1, 1:1),
    %encoder.10.layers.14.0.bias : Float(512:1),
    %encoder.11.layers.0.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.11.layers.0.0.bias : Float(512:1),
    %encoder.11.layers.4._se_reduce.weight : Float(102:512, 512:1, 1:1),
    %encoder.11.layers.4._se_reduce.bias : Float(102:1),
    %encoder.11.layers.4._se_expand.weight : Float(512:102, 102:1, 1:1),
    %encoder.11.layers.4._se_expand.bias : Float(512:1),
    %encoder.11.layers.5.0.weight : Float(512:512, 512:1, 1:1),
    %encoder.11.layers.5.0.bias : Float(512:1),
    %encoder.11.layers.9.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.11.layers.9.0.bias : Float(512:1),
    %encoder.11.layers.13._se_reduce.weight : Float(102:512, 512:1, 1:1),
    %encoder.11.layers.13._se_reduce.bias : Float(102:1),
    %encoder.11.layers.13._se_expand.weight : Float(512:102, 102:1, 1:1),
    %encoder.11.layers.13._se_expand.bias : Float(512:1),
    %encoder.11.layers.14.0.weight : Float(512:512, 512:1, 1:1),
    %encoder.11.layers.14.0.bias : Float(512:1),
    %encoder.12.layers.0.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.12.layers.0.0.bias : Float(512:1),
    %encoder.12.layers.4._se_reduce.weight : Float(102:512, 512:1, 1:1),
    %encoder.12.layers.4._se_reduce.bias : Float(102:1),
    %encoder.12.layers.4._se_expand.weight : Float(512:102, 102:1, 1:1),
    %encoder.12.layers.4._se_expand.bias : Float(512:1),
    %encoder.12.layers.5.0.weight : Float(512:512, 512:1, 1:1),
    %encoder.12.layers.5.0.bias : Float(512:1),
    %encoder.12.layers.9.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.12.layers.9.0.bias : Float(512:1),
    %encoder.12.layers.13._se_reduce.weight : Float(102:512, 512:1, 1:1),
    %encoder.12.layers.13._se_reduce.bias : Float(102:1),
    %encoder.12.layers.13._se_expand.weight : Float(512:102, 102:1, 1:1),
    %encoder.12.layers.13._se_expand.bias : Float(512:1),
    %encoder.12.layers.14.0.weight : Float(512:512, 512:1, 1:1),
    %encoder.12.layers.14.0.bias : Float(512:1),
    %encoder.13.layers.0.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.13.layers.0.0.bias : Float(512:1),
    %encoder.13.layers.4._se_reduce.weight : Float(102:512, 512:1, 1:1),
    %encoder.13.layers.4._se_reduce.bias : Float(102:1),
    %encoder.13.layers.4._se_expand.weight : Float(512:102, 102:1, 1:1),
    %encoder.13.layers.4._se_expand.bias : Float(512:1),
    %encoder.13.layers.5.0.weight : Float(512:512, 512:1, 1:1),
    %encoder.13.layers.5.0.bias : Float(512:1),
    %encoder.13.layers.9.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.13.layers.9.0.bias : Float(512:1),
    %encoder.13.layers.13._se_reduce.weight : Float(102:512, 512:1, 1:1),
    %encoder.13.layers.13._se_reduce.bias : Float(102:1),
    %encoder.13.layers.13._se_expand.weight : Float(512:102, 102:1, 1:1),
    %encoder.13.layers.13._se_expand.bias : Float(512:1),
    %encoder.13.layers.14.0.weight : Float(512:512, 512:1, 1:1),
    %encoder.13.layers.14.0.bias : Float(512:1),
    %encoder.14.layers.0.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.14.layers.0.0.bias : Float(512:1),
    %encoder.14.layers.4.0.weight : Float(512:512, 512:1, 1:1),
    %encoder.14.layers.4.0.bias : Float(512:1),
    %encoder.15.layers.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.15.layers.1.weight : Float(512:1),
    %encoder.15.layers.1.bias : Float(512:1),
    %encoder.15.layers.1.running_mean : Float(512:1),
    %encoder.15.layers.1.running_var : Float(512:1),
    %encoder.15.layers.4._se_reduce.weight : Float(102:512, 512:1, 1:1),
    %encoder.15.layers.4._se_reduce.bias : Float(102:1),
    %encoder.15.layers.4._se_expand.weight : Float(512:102, 102:1, 1:1),
    %encoder.15.layers.4._se_expand.bias : Float(512:1),
    %encoder.15.layers.5.weight : Float(512:512, 512:1, 1:1),
    %encoder.15.layers.5.bias : Float(512:1),
    %encoder.15.layers.6.weight : Float(512:1),
    %encoder.15.layers.6.bias : Float(512:1),
    %encoder.15.layers.6.running_mean : Float(512:1),
    %encoder.15.layers.6.running_var : Float(512:1),
    %encoder.16.layers.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.16.layers.1.weight : Float(512:1),
    %encoder.16.layers.1.bias : Float(512:1),
    %encoder.16.layers.1.running_mean : Float(512:1),
    %encoder.16.layers.1.running_var : Float(512:1),
    %encoder.16.layers.4._se_reduce.weight : Float(102:512, 512:1, 1:1),
    %encoder.16.layers.4._se_reduce.bias : Float(102:1),
    %encoder.16.layers.4._se_expand.weight : Float(512:102, 102:1, 1:1),
    %encoder.16.layers.4._se_expand.bias : Float(512:1),
    %encoder.16.layers.5.weight : Float(512:512, 512:1, 1:1),
    %encoder.16.layers.5.bias : Float(512:1),
    %encoder.16.layers.6.weight : Float(512:1),
    %encoder.16.layers.6.bias : Float(512:1),
    %encoder.16.layers.6.running_mean : Float(512:1),
    %encoder.16.layers.6.running_var : Float(512:1),
    %encoder.17.layers.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.17.layers.1.weight : Float(512:1),
    %encoder.17.layers.1.bias : Float(512:1),
    %encoder.17.layers.1.running_mean : Float(512:1),
    %encoder.17.layers.1.running_var : Float(512:1),
    %encoder.17.layers.4._se_reduce.weight : Float(102:512, 512:1, 1:1),
    %encoder.17.layers.4._se_reduce.bias : Float(102:1),
    %encoder.17.layers.4._se_expand.weight : Float(512:102, 102:1, 1:1),
    %encoder.17.layers.4._se_expand.bias : Float(512:1),
    %encoder.17.layers.5.weight : Float(512:512, 512:1, 1:1),
    %encoder.17.layers.5.bias : Float(512:1),
    %encoder.17.layers.6.weight : Float(512:1),
    %encoder.17.layers.6.bias : Float(512:1),
    %encoder.17.layers.6.running_mean : Float(512:1),
    %encoder.17.layers.6.running_var : Float(512:1),
    %encoder.18.layers.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.18.layers.1.weight : Float(512:1),
    %encoder.18.layers.1.bias : Float(512:1),
    %encoder.18.layers.1.running_mean : Float(512:1),
    %encoder.18.layers.1.running_var : Float(512:1),
    %encoder.18.layers.4._se_reduce.weight : Float(102:512, 512:1, 1:1),
    %encoder.18.layers.4._se_reduce.bias : Float(102:1),
    %encoder.18.layers.4._se_expand.weight : Float(512:102, 102:1, 1:1),
    %encoder.18.layers.4._se_expand.bias : Float(512:1),
    %encoder.18.layers.5.weight : Float(512:512, 512:1, 1:1),
    %encoder.18.layers.5.bias : Float(512:1),
    %encoder.18.layers.6.weight : Float(512:1),
    %encoder.18.layers.6.bias : Float(512:1),
    %encoder.18.layers.6.running_mean : Float(512:1),
    %encoder.18.layers.6.running_var : Float(512:1),
    %encoder.19.layers.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.19.layers.1.weight : Float(512:1),
    %encoder.19.layers.1.bias : Float(512:1),
    %encoder.19.layers.1.running_mean : Float(512:1),
    %encoder.19.layers.1.running_var : Float(512:1),
    %encoder.19.layers.4._se_reduce.weight : Float(102:512, 512:1, 1:1),
    %encoder.19.layers.4._se_reduce.bias : Float(102:1),
    %encoder.19.layers.4._se_expand.weight : Float(512:102, 102:1, 1:1),
    %encoder.19.layers.4._se_expand.bias : Float(512:1),
    %encoder.19.layers.5.weight : Float(512:512, 512:1, 1:1),
    %encoder.19.layers.5.bias : Float(512:1),
    %encoder.19.layers.6.weight : Float(512:1),
    %encoder.19.layers.6.bias : Float(512:1),
    %encoder.19.layers.6.running_mean : Float(512:1),
    %encoder.19.layers.6.running_var : Float(512:1),
    %encoder.20.layers.0.weight : Float(512:448, 64:7, 7:1),
    %encoder.20.layers.1.weight : Float(512:1),
    %encoder.20.layers.1.bias : Float(512:1),
    %encoder.20.layers.1.running_mean : Float(512:1),
    %encoder.20.layers.1.running_var : Float(512:1),
    %encoder.20.layers.4._se_reduce.weight : Float(102:512, 512:1, 1:1),
    %encoder.20.layers.4._se_reduce.bias : Float(102:1),
    %encoder.20.layers.4._se_expand.weight : Float(512:102, 102:1, 1:1),
    %encoder.20.layers.4._se_expand.bias : Float(512:1),
    %encoder.20.layers.5.weight : Float(512:512, 512:1, 1:1),
    %encoder.20.layers.5.bias : Float(512:1),
    %encoder.20.layers.6.weight : Float(512:1),
    %encoder.20.layers.6.bias : Float(512:1),
    %encoder.20.layers.6.running_mean : Float(512:1),
    %encoder.20.layers.6.running_var : Float(512:1),
    %decoder.layers.0.self_attn.in_proj_bias : Float(1536:1),
    %decoder.layers.0.self_attn.out_proj.bias : Float(512:1),
    %decoder.layers.0.linear1.bias : Float(512:1),
    %decoder.layers.0.linear2.bias : Float(512:1),
    %decoder.layers.0.norm1.weight : Float(512:1),
    %decoder.layers.0.norm1.bias : Float(512:1),
    %decoder.layers.0.norm2.weight : Float(512:1),
    %decoder.layers.0.norm2.bias : Float(512:1),
    %decoder.layers.1.self_attn.in_proj_bias : Float(1536:1),
    %decoder.layers.1.self_attn.out_proj.bias : Float(512:1),
    %decoder.layers.1.linear1.bias : Float(512:1),
    %decoder.layers.1.linear2.bias : Float(512:1),
    %decoder.layers.1.norm1.weight : Float(512:1),
    %decoder.layers.1.norm1.bias : Float(512:1),
    %decoder.layers.1.norm2.weight : Float(512:1),
    %decoder.layers.1.norm2.bias : Float(512:1),
    %decoder.layers.2.self_attn.in_proj_bias : Float(1536:1),
    %decoder.layers.2.self_attn.out_proj.bias : Float(512:1),
    %decoder.layers.2.linear1.bias : Float(512:1),
    %decoder.layers.2.linear2.bias : Float(512:1),
    %decoder.layers.2.norm1.weight : Float(512:1),
    %decoder.layers.2.norm1.bias : Float(512:1),
    %decoder.layers.2.norm2.weight : Float(512:1),
    %decoder.layers.2.norm2.bias : Float(512:1),
    %decoder.layers.3.self_attn.in_proj_bias : Float(1536:1),
    %decoder.layers.3.self_attn.out_proj.bias : Float(512:1),
    %decoder.layers.3.linear1.bias : Float(512:1),
    %decoder.layers.3.linear2.bias : Float(512:1),
    %decoder.layers.3.norm1.weight : Float(512:1),
    %decoder.layers.3.norm1.bias : Float(512:1),
    %decoder.layers.3.norm2.weight : Float(512:1),
    %decoder.layers.3.norm2.bias : Float(512:1),
    %decoder.layers.4.self_attn.in_proj_bias : Float(1536:1),
    %decoder.layers.4.self_attn.out_proj.bias : Float(512:1),
    %decoder.layers.4.linear1.bias : Float(512:1),
    %decoder.layers.4.linear2.bias : Float(512:1),
    %decoder.layers.4.norm1.weight : Float(512:1),
    %decoder.layers.4.norm1.bias : Float(512:1),
    %decoder.layers.4.norm2.weight : Float(512:1),
    %decoder.layers.4.norm2.bias : Float(512:1),
    %decoder.layers.5.self_attn.in_proj_bias : Float(1536:1),
    %decoder.layers.5.self_attn.out_proj.bias : Float(512:1),
    %decoder.layers.5.linear1.bias : Float(512:1),
    %decoder.layers.5.linear2.bias : Float(512:1),
    %decoder.layers.5.norm1.weight : Float(512:1),
    %decoder.layers.5.norm1.bias : Float(512:1),
    %decoder.layers.5.norm2.weight : Float(512:1),
    %decoder.layers.5.norm2.bias : Float(512:1),
    %decoder.layers.6.self_attn.in_proj_bias : Float(1536:1),
    %decoder.layers.6.self_attn.out_proj.bias : Float(512:1),
    %decoder.layers.6.linear1.bias : Float(512:1),
    %decoder.layers.6.linear2.bias : Float(512:1),
    %decoder.layers.6.norm1.weight : Float(512:1),
    %decoder.layers.6.norm1.bias : Float(512:1),
    %decoder.layers.6.norm2.weight : Float(512:1),
    %decoder.layers.6.norm2.bias : Float(512:1),
    %decoder.layers.7.self_attn.in_proj_bias : Float(1536:1),
    %decoder.layers.7.self_attn.out_proj.bias : Float(512:1),
    %decoder.layers.7.linear1.bias : Float(512:1),
    %decoder.layers.7.linear2.bias : Float(512:1),
    %decoder.layers.7.norm1.weight : Float(512:1),
    %decoder.layers.7.norm1.bias : Float(512:1),
    %decoder.layers.7.norm2.weight : Float(512:1),
    %decoder.layers.7.norm2.bias : Float(512:1),
    %decoder.layers.8.self_attn.in_proj_bias : Float(1536:1),
    %decoder.layers.8.self_attn.out_proj.bias : Float(512:1),
    %decoder.layers.8.linear1.bias : Float(512:1),
    %decoder.layers.8.linear2.bias : Float(512:1),
    %decoder.layers.8.norm1.weight : Float(512:1),
    %decoder.layers.8.norm1.bias : Float(512:1),
    %decoder.layers.8.norm2.weight : Float(512:1),
    %decoder.layers.8.norm2.bias : Float(512:1),
    %decoder.layers.9.self_attn.in_proj_bias : Float(1536:1),
    %decoder.layers.9.self_attn.out_proj.bias : Float(512:1),
    %decoder.layers.9.linear1.bias : Float(512:1),
    %decoder.layers.9.linear2.bias : Float(512:1),
    %decoder.layers.9.norm1.weight : Float(512:1),
    %decoder.layers.9.norm1.bias : Float(512:1),
    %decoder.layers.9.norm2.weight : Float(512:1),
    %decoder.layers.9.norm2.bias : Float(512:1),
    %decoder.layers.10.self_attn.in_proj_bias : Float(1536:1),
    %decoder.layers.10.self_attn.out_proj.bias : Float(512:1),
    %decoder.layers.10.linear1.bias : Float(512:1),
    %decoder.layers.10.linear2.bias : Float(512:1),
    %decoder.layers.10.norm1.weight : Float(512:1),
    %decoder.layers.10.norm1.bias : Float(512:1),
    %decoder.layers.10.norm2.weight : Float(512:1),
    %decoder.layers.10.norm2.bias : Float(512:1),
    %decoder.layers.11.self_attn.in_proj_bias : Float(1536:1),
    %decoder.layers.11.self_attn.out_proj.bias : Float(512:1),
    %decoder.layers.11.linear1.bias : Float(512:1),
    %decoder.layers.11.linear2.bias : Float(512:1),
    %decoder.layers.11.norm1.weight : Float(512:1),
    %decoder.layers.11.norm1.bias : Float(512:1),
    %decoder.layers.11.norm2.weight : Float(512:1),
    %decoder.layers.11.norm2.bias : Float(512:1),
    %decoder.layers.12.self_attn.in_proj_bias : Float(1536:1),
    %decoder.layers.12.self_attn.out_proj.bias : Float(512:1),
    %decoder.layers.12.linear1.bias : Float(512:1),
    %decoder.layers.12.linear2.bias : Float(512:1),
    %decoder.layers.12.norm1.weight : Float(512:1),
    %decoder.layers.12.norm1.bias : Float(512:1),
    %decoder.layers.12.norm2.weight : Float(512:1),
    %decoder.layers.12.norm2.bias : Float(512:1),
    %decoder.layers.13.self_attn.in_proj_bias : Float(1536:1),
    %decoder.layers.13.self_attn.out_proj.bias : Float(512:1),
    %decoder.layers.13.linear1.bias : Float(512:1),
    %decoder.layers.13.linear2.bias : Float(512:1),
    %decoder.layers.13.norm1.weight : Float(512:1),
    %decoder.layers.13.norm1.bias : Float(512:1),
    %decoder.layers.13.norm2.weight : Float(512:1),
    %decoder.layers.13.norm2.bias : Float(512:1),
    %decoder.layers.14.self_attn.in_proj_bias : Float(1536:1),
    %decoder.layers.14.self_attn.out_proj.bias : Float(512:1),
    %decoder.layers.14.linear1.bias : Float(512:1),
    %decoder.layers.14.linear2.bias : Float(512:1),
    %decoder.layers.14.norm1.weight : Float(512:1),
    %decoder.layers.14.norm1.bias : Float(512:1),
    %decoder.layers.14.norm2.weight : Float(512:1),
    %decoder.layers.14.norm2.bias : Float(512:1),
    %decoder.layers.15.self_attn.in_proj_bias : Float(1536:1),
    %decoder.layers.15.self_attn.out_proj.bias : Float(512:1),
    %decoder.layers.15.linear1.bias : Float(512:1),
    %decoder.layers.15.linear2.bias : Float(512:1),
    %decoder.layers.15.norm1.weight : Float(512:1),
    %decoder.layers.15.norm1.bias : Float(512:1),
    %decoder.layers.15.norm2.weight : Float(512:1),
    %decoder.layers.15.norm2.bias : Float(512:1),
    %fc.weight : Float(999:512, 512:1, 1:1),
    %fc.bias : Float(999:1),
    %2460 : Long(1:1),
    %2461 : Float(),
    %2462 : Float(),
    %2463 : Float(512:1, 1536:512),
    %2464 : Long(1:1),
    %2465 : Long(1:1),
    %2466 : Float(512:1, 512:512),
    %2467 : Float(),
    %2468 : Float(512:1, 512:512),
    %2469 : Float(512:1, 512:512),
    %2470 : Float(),
    %2471 : Float(512:1, 1536:512),
    %2472 : Long(1:1),
    %2473 : Long(1:1),
    %2474 : Float(512:1, 512:512),
    %2475 : Float(),
    %2476 : Float(512:1, 512:512),
    %2477 : Float(512:1, 512:512),
    %2478 : Float(),
    %2479 : Float(512:1, 1536:512),
    %2480 : Long(1:1),
    %2481 : Long(1:1),
    %2482 : Float(512:1, 512:512),
    %2483 : Float(),
    %2484 : Float(512:1, 512:512),
    %2485 : Float(512:1, 512:512),
    %2486 : Float(),
    %2487 : Float(512:1, 1536:512),
    %2488 : Long(1:1),
    %2489 : Long(1:1),
    %2490 : Float(512:1, 512:512),
    %2491 : Float(),
    %2492 : Float(512:1, 512:512),
    %2493 : Float(512:1, 512:512),
    %2494 : Float(),
    %2495 : Float(512:1, 1536:512),
    %2496 : Long(1:1),
    %2497 : Long(1:1),
    %2498 : Float(512:1, 512:512),
    %2499 : Float(),
    %2500 : Float(512:1, 512:512),
    %2501 : Float(512:1, 512:512),
    %2502 : Float(),
    %2503 : Float(512:1, 1536:512),
    %2504 : Long(1:1),
    %2505 : Long(1:1),
    %2506 : Float(512:1, 512:512),
    %2507 : Float(),
    %2508 : Float(512:1, 512:512),
    %2509 : Float(512:1, 512:512),
    %2510 : Float(),
    %2511 : Float(512:1, 1536:512),
    %2512 : Long(1:1),
    %2513 : Long(1:1),
    %2514 : Float(512:1, 512:512),
    %2515 : Float(),
    %2516 : Float(512:1, 512:512),
    %2517 : Float(512:1, 512:512),
    %2518 : Float(),
    %2519 : Float(512:1, 1536:512),
    %2520 : Long(1:1),
    %2521 : Long(1:1),
    %2522 : Float(512:1, 512:512),
    %2523 : Float(),
    %2524 : Float(512:1, 512:512),
    %2525 : Float(512:1, 512:512),
    %2526 : Float(),
    %2527 : Float(512:1, 1536:512),
    %2528 : Long(1:1),
    %2529 : Long(1:1),
    %2530 : Float(512:1, 512:512),
    %2531 : Float(),
    %2532 : Float(512:1, 512:512),
    %2533 : Float(512:1, 512:512),
    %2534 : Float(),
    %2535 : Float(512:1, 1536:512),
    %2536 : Long(1:1),
    %2537 : Long(1:1),
    %2538 : Float(512:1, 512:512),
    %2539 : Float(),
    %2540 : Float(512:1, 512:512),
    %2541 : Float(512:1, 512:512),
    %2542 : Float(),
    %2543 : Float(512:1, 1536:512),
    %2544 : Long(1:1),
    %2545 : Long(1:1),
    %2546 : Float(512:1, 512:512),
    %2547 : Float(),
    %2548 : Float(512:1, 512:512),
    %2549 : Float(512:1, 512:512),
    %2550 : Float(),
    %2551 : Float(512:1, 1536:512),
    %2552 : Long(1:1),
    %2553 : Long(1:1),
    %2554 : Float(512:1, 512:512),
    %2555 : Float(),
    %2556 : Float(512:1, 512:512),
    %2557 : Float(512:1, 512:512),
    %2558 : Float(),
    %2559 : Float(512:1, 1536:512),
    %2560 : Long(1:1),
    %2561 : Long(1:1),
    %2562 : Float(512:1, 512:512),
    %2563 : Float(),
    %2564 : Float(512:1, 512:512),
    %2565 : Float(512:1, 512:512),
    %2566 : Float(),
    %2567 : Float(512:1, 1536:512),
    %2568 : Long(1:1),
    %2569 : Long(1:1),
    %2570 : Float(512:1, 512:512),
    %2571 : Float(),
    %2572 : Float(512:1, 512:512),
    %2573 : Float(512:1, 512:512),
    %2574 : Float(),
    %2575 : Float(512:1, 1536:512),
    %2576 : Long(1:1),
    %2577 : Long(1:1),
    %2578 : Float(512:1, 512:512),
    %2579 : Float(),
    %2580 : Float(512:1, 512:512),
    %2581 : Float(512:1, 512:512),
    %2582 : Float(),
    %2583 : Float(512:1, 1536:512),
    %2584 : Long(1:1),
    %2585 : Long(1:1),
    %2586 : Float(512:1, 512:512),
    %2587 : Float(),
    %2588 : Float(512:1, 512:512),
    %2589 : Float(512:1, 512:512),
    %2590 : Float()):
%504 : Tensor = onnx::Shape(%input)
%505 : Tensor = onnx::Constant[value={0}]()
%506 : Long() = onnx::Gather[axis=0](%504, %505) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/pytorch_stft.py:49:0
%507 : Tensor = onnx::Shape(%input)
%508 : Tensor = onnx::Constant[value={1}]()
%509 : Long() = onnx::Gather[axis=0](%507, %508) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/pytorch_stft.py:50:0
%511 : Tensor = onnx::Unsqueeze[axes=[0]](%506)
%513 : Tensor = onnx::Unsqueeze[axes=[0]](%509)
%514 : Tensor = onnx::Concat[axis=0](%511, %2460, %513)
%515 : Float(1:73728, 1:73728, 73728:1) = onnx::Reshape(%input, %514) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/pytorch_stft.py:55:0
%516 : Float(1:73728, 1:73728, 1:73728, 73728:1) = onnx::Unsqueeze[axes=[1]](%515) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/pytorch_stft.py:57:0
%517 : Float(1:74048, 1:74048, 1:74048, 74048:1) = onnx::Pad[mode="reflect", pads=[0, 0, 0, 160, 0, 0, 0, 160]](%516) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3569:0
%518 : Float(1:74048, 1:74048, 74048:1) = onnx::Squeeze[axes=[1]](%517) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/pytorch_stft.py:60:0
%519 : Float(1:148442, 322:461, 461:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[320], pads=[0, 0], strides=[160]](%518, %stft.forward_basis) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/pytorch_stft.py:70:0
%520 : Tensor = onnx::Constant[value={1}]()
%521 : Tensor = onnx::Constant[value={0}]()
%522 : Tensor = onnx::Constant[value={161}]()
%523 : Tensor = onnx::Constant[value={1}]()
%524 : Float(1:148442, 161:461, 461:1) = onnx::Slice(%519, %521, %522, %520, %523) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/pytorch_stft.py:69:0
%525 : Tensor = onnx::Constant[value={1}]()
%526 : Tensor = onnx::Constant[value={161}]()
%527 : Tensor = onnx::Constant[value={9223372036854775807}]()
%528 : Tensor = onnx::Constant[value={1}]()
%529 : Float(1:148442, 161:461, 461:1) = onnx::Slice(%519, %526, %527, %525, %528) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/pytorch_stft.py:70:0
%532 : Float(1:148442, 161:461, 461:1) = onnx::Pow(%524, %2461) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/pytorch_stft.py:72:0
%535 : Float(1:148442, 161:461, 461:1) = onnx::Pow(%529, %2462) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/pytorch_stft.py:72:0
%536 : Float(1:74221, 161:461, 461:1) = onnx::Add(%532, %535) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/pytorch_stft.py:72:0
%537 : Float(1:74221, 161:461, 461:1) = onnx::Sqrt(%536) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/pytorch_stft.py:72:0
%538 : Float() = onnx::Constant[value={1.04858e+06}]()
%539 : Float(1:74221, 161:461, 461:1) = onnx::Mul(%537, %538)
%540 : Float() = onnx::Constant[value={1}]()
%541 : FloatTensor = onnx::Add(%540, %539)
%542 : Float(1:74221, 161:461, 461:1) = onnx::Log(%541) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model.py:444:0
%543 : Float(1:461, 1:461, 461:1) = onnx::ReduceMean[axes=[1], keepdims=1](%542) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model.py:447:0
%544 : Float(1:621, 1:621, 621:1) = onnx::Pad[mode="reflect", pads=[0, 0, 80, 0, 0, 80]](%543) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3558:0
%545 : Float(1:461, 1:461, 461:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[161], pads=[0, 0], strides=[1]](%544, %audio_normalize.filter_) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model.py:449:0
%546 : Float(1:1, 1:1, 1:1) = onnx::ReduceMean[axes=[-1], keepdims=1](%545) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model.py:450:0
%547 : Float(1:1, 1:1, 1:1) = onnx::Neg(%546) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model.py:451:0
%548 : Float(1:74221, 161:461, 461:1) = onnx::Add(%542, %547) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model.py:451:0
%549 : Float(1:118272, 512:231, 231:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[7], pads=[3, 3], strides=[2]](%548, %encoder.0.layers.0.0.weight, %encoder.0.layers.0.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%550 : Float(1:118272, 512:231, 231:1) = onnx::Relu(%549) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%551 : Float(1:118272, 512:231, 231:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%550, %encoder.0.layers.4.0.weight, %encoder.0.layers.4.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%552 : Float(1:118272, 512:231, 231:1) = onnx::Relu(%551) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%553 : Float(1:118272, 512:231, 231:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[1]](%552, %encoder.1.layers.0.0.weight, %encoder.1.layers.0.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%554 : Float(1:118272, 512:231, 231:1) = onnx::Relu(%553) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%555 : Float(1:512, 512:1) = onnx::ReduceMean[axes=[2], keepdims=0](%554) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%556 : Float(1:512, 512:1, 1:1) = onnx::Unsqueeze[axes=[2]](%555) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%557 : Float(1:102, 102:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%556, %encoder.1.layers.4._se_reduce.weight, %encoder.1.layers.4._se_reduce.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%558 : Float(1:102, 102:1, 1:1) = onnx::Sigmoid(%557) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%559 : Float(1:102, 102:1, 1:1) = onnx::Mul(%557, %558) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:28:0
%560 : Float(1:512, 512:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%559, %encoder.1.layers.4._se_expand.weight, %encoder.1.layers.4._se_expand.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%561 : Float(1:512, 512:1, 1:1) = onnx::Sigmoid(%560) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%562 : Float(1:118272, 512:231, 231:1) = onnx::Mul(%561, %554) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:59:0
%563 : Float(1:118272, 512:231, 231:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%562, %encoder.1.layers.5.0.weight, %encoder.1.layers.5.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%564 : Float(1:118272, 512:231, 231:1) = onnx::Relu(%563) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%565 : Float(1:118272, 512:231, 231:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[1]](%564, %encoder.1.layers.9.0.weight, %encoder.1.layers.9.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%566 : Float(1:118272, 512:231, 231:1) = onnx::Relu(%565) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%567 : Float(1:512, 512:1) = onnx::ReduceMean[axes=[2], keepdims=0](%566) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%568 : Float(1:512, 512:1, 1:1) = onnx::Unsqueeze[axes=[2]](%567) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%569 : Float(1:102, 102:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%568, %encoder.1.layers.13._se_reduce.weight, %encoder.1.layers.13._se_reduce.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%570 : Float(1:102, 102:1, 1:1) = onnx::Sigmoid(%569) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%571 : Float(1:102, 102:1, 1:1) = onnx::Mul(%569, %570) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:28:0
%572 : Float(1:512, 512:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%571, %encoder.1.layers.13._se_expand.weight, %encoder.1.layers.13._se_expand.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%573 : Float(1:512, 512:1, 1:1) = onnx::Sigmoid(%572) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%574 : Float(1:118272, 512:231, 231:1) = onnx::Mul(%573, %566) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:59:0
%575 : Float(1:118272, 512:231, 231:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%574, %encoder.1.layers.14.0.weight, %encoder.1.layers.14.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%576 : Float(1:118272, 512:231, 231:1) = onnx::Relu(%575) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%577 : Float(1:118272, 512:231, 231:1) = onnx::Add(%576, %552) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:45:0
%578 : Float(1:118272, 512:231, 231:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[1]](%577, %encoder.2.layers.0.0.weight, %encoder.2.layers.0.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%579 : Float(1:118272, 512:231, 231:1) = onnx::Relu(%578) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%580 : Float(1:512, 512:1) = onnx::ReduceMean[axes=[2], keepdims=0](%579) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%581 : Float(1:512, 512:1, 1:1) = onnx::Unsqueeze[axes=[2]](%580) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%582 : Float(1:102, 102:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%581, %encoder.2.layers.4._se_reduce.weight, %encoder.2.layers.4._se_reduce.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%583 : Float(1:102, 102:1, 1:1) = onnx::Sigmoid(%582) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%584 : Float(1:102, 102:1, 1:1) = onnx::Mul(%582, %583) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:28:0
%585 : Float(1:512, 512:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%584, %encoder.2.layers.4._se_expand.weight, %encoder.2.layers.4._se_expand.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%586 : Float(1:512, 512:1, 1:1) = onnx::Sigmoid(%585) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%587 : Float(1:118272, 512:231, 231:1) = onnx::Mul(%586, %579) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:59:0
%588 : Float(1:118272, 512:231, 231:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%587, %encoder.2.layers.5.0.weight, %encoder.2.layers.5.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%589 : Float(1:118272, 512:231, 231:1) = onnx::Relu(%588) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%590 : Float(1:118272, 512:231, 231:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[1]](%589, %encoder.2.layers.9.0.weight, %encoder.2.layers.9.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%591 : Float(1:118272, 512:231, 231:1) = onnx::Relu(%590) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%592 : Float(1:512, 512:1) = onnx::ReduceMean[axes=[2], keepdims=0](%591) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%593 : Float(1:512, 512:1, 1:1) = onnx::Unsqueeze[axes=[2]](%592) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%594 : Float(1:102, 102:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%593, %encoder.2.layers.13._se_reduce.weight, %encoder.2.layers.13._se_reduce.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%595 : Float(1:102, 102:1, 1:1) = onnx::Sigmoid(%594) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%596 : Float(1:102, 102:1, 1:1) = onnx::Mul(%594, %595) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:28:0
%597 : Float(1:512, 512:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%596, %encoder.2.layers.13._se_expand.weight, %encoder.2.layers.13._se_expand.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%598 : Float(1:512, 512:1, 1:1) = onnx::Sigmoid(%597) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%599 : Float(1:118272, 512:231, 231:1) = onnx::Mul(%598, %591) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:59:0
%600 : Float(1:118272, 512:231, 231:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%599, %encoder.2.layers.14.0.weight, %encoder.2.layers.14.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%601 : Float(1:118272, 512:231, 231:1) = onnx::Relu(%600) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%602 : Float(1:118272, 512:231, 231:1) = onnx::Add(%601, %577) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:45:0
%603 : Float(1:118272, 512:231, 231:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[1]](%602, %encoder.3.layers.0.0.weight, %encoder.3.layers.0.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%604 : Float(1:118272, 512:231, 231:1) = onnx::Relu(%603) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%605 : Float(1:512, 512:1) = onnx::ReduceMean[axes=[2], keepdims=0](%604) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%606 : Float(1:512, 512:1, 1:1) = onnx::Unsqueeze[axes=[2]](%605) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%607 : Float(1:102, 102:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%606, %encoder.3.layers.4._se_reduce.weight, %encoder.3.layers.4._se_reduce.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%608 : Float(1:102, 102:1, 1:1) = onnx::Sigmoid(%607) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%609 : Float(1:102, 102:1, 1:1) = onnx::Mul(%607, %608) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:28:0
%610 : Float(1:512, 512:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%609, %encoder.3.layers.4._se_expand.weight, %encoder.3.layers.4._se_expand.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%611 : Float(1:512, 512:1, 1:1) = onnx::Sigmoid(%610) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%612 : Float(1:118272, 512:231, 231:1) = onnx::Mul(%611, %604) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:59:0
%613 : Float(1:118272, 512:231, 231:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%612, %encoder.3.layers.5.0.weight, %encoder.3.layers.5.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%614 : Float(1:118272, 512:231, 231:1) = onnx::Relu(%613) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%615 : Float(1:118272, 512:231, 231:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[1]](%614, %encoder.3.layers.9.0.weight, %encoder.3.layers.9.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%616 : Float(1:118272, 512:231, 231:1) = onnx::Relu(%615) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%617 : Float(1:512, 512:1) = onnx::ReduceMean[axes=[2], keepdims=0](%616) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%618 : Float(1:512, 512:1, 1:1) = onnx::Unsqueeze[axes=[2]](%617) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%619 : Float(1:102, 102:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%618, %encoder.3.layers.13._se_reduce.weight, %encoder.3.layers.13._se_reduce.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%620 : Float(1:102, 102:1, 1:1) = onnx::Sigmoid(%619) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%621 : Float(1:102, 102:1, 1:1) = onnx::Mul(%619, %620) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:28:0
%622 : Float(1:512, 512:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%621, %encoder.3.layers.13._se_expand.weight, %encoder.3.layers.13._se_expand.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%623 : Float(1:512, 512:1, 1:1) = onnx::Sigmoid(%622) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%624 : Float(1:118272, 512:231, 231:1) = onnx::Mul(%623, %616) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:59:0
%625 : Float(1:118272, 512:231, 231:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%624, %encoder.3.layers.14.0.weight, %encoder.3.layers.14.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%626 : Float(1:118272, 512:231, 231:1) = onnx::Relu(%625) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%627 : Float(1:118272, 512:231, 231:1) = onnx::Add(%626, %602) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:45:0
%628 : Float(1:118272, 512:231, 231:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[1]](%627, %encoder.4.layers.0.0.weight, %encoder.4.layers.0.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%629 : Float(1:118272, 512:231, 231:1) = onnx::Relu(%628) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%630 : Float(1:512, 512:1) = onnx::ReduceMean[axes=[2], keepdims=0](%629) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%631 : Float(1:512, 512:1, 1:1) = onnx::Unsqueeze[axes=[2]](%630) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%632 : Float(1:102, 102:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%631, %encoder.4.layers.4._se_reduce.weight, %encoder.4.layers.4._se_reduce.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%633 : Float(1:102, 102:1, 1:1) = onnx::Sigmoid(%632) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%634 : Float(1:102, 102:1, 1:1) = onnx::Mul(%632, %633) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:28:0
%635 : Float(1:512, 512:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%634, %encoder.4.layers.4._se_expand.weight, %encoder.4.layers.4._se_expand.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%636 : Float(1:512, 512:1, 1:1) = onnx::Sigmoid(%635) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%637 : Float(1:118272, 512:231, 231:1) = onnx::Mul(%636, %629) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:59:0
%638 : Float(1:118272, 512:231, 231:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%637, %encoder.4.layers.5.0.weight, %encoder.4.layers.5.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%639 : Float(1:118272, 512:231, 231:1) = onnx::Relu(%638) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%640 : Float(1:118272, 512:231, 231:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[1]](%639, %encoder.4.layers.9.0.weight, %encoder.4.layers.9.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%641 : Float(1:118272, 512:231, 231:1) = onnx::Relu(%640) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%642 : Float(1:512, 512:1) = onnx::ReduceMean[axes=[2], keepdims=0](%641) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%643 : Float(1:512, 512:1, 1:1) = onnx::Unsqueeze[axes=[2]](%642) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%644 : Float(1:102, 102:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%643, %encoder.4.layers.13._se_reduce.weight, %encoder.4.layers.13._se_reduce.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%645 : Float(1:102, 102:1, 1:1) = onnx::Sigmoid(%644) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%646 : Float(1:102, 102:1, 1:1) = onnx::Mul(%644, %645) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:28:0
%647 : Float(1:512, 512:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%646, %encoder.4.layers.13._se_expand.weight, %encoder.4.layers.13._se_expand.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%648 : Float(1:512, 512:1, 1:1) = onnx::Sigmoid(%647) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%649 : Float(1:118272, 512:231, 231:1) = onnx::Mul(%648, %641) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:59:0
%650 : Float(1:118272, 512:231, 231:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%649, %encoder.4.layers.14.0.weight, %encoder.4.layers.14.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%651 : Float(1:118272, 512:231, 231:1) = onnx::Relu(%650) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%652 : Float(1:118272, 512:231, 231:1) = onnx::Add(%651, %627) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:45:0
%653 : Float(1:118272, 512:231, 231:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[1]](%652, %encoder.5.layers.0.0.weight, %encoder.5.layers.0.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%654 : Float(1:118272, 512:231, 231:1) = onnx::Relu(%653) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%655 : Float(1:512, 512:1) = onnx::ReduceMean[axes=[2], keepdims=0](%654) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%656 : Float(1:512, 512:1, 1:1) = onnx::Unsqueeze[axes=[2]](%655) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%657 : Float(1:102, 102:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%656, %encoder.5.layers.4._se_reduce.weight, %encoder.5.layers.4._se_reduce.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%658 : Float(1:102, 102:1, 1:1) = onnx::Sigmoid(%657) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%659 : Float(1:102, 102:1, 1:1) = onnx::Mul(%657, %658) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:28:0
%660 : Float(1:512, 512:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%659, %encoder.5.layers.4._se_expand.weight, %encoder.5.layers.4._se_expand.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%661 : Float(1:512, 512:1, 1:1) = onnx::Sigmoid(%660) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%662 : Float(1:118272, 512:231, 231:1) = onnx::Mul(%661, %654) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:59:0
%663 : Float(1:118272, 512:231, 231:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%662, %encoder.5.layers.5.0.weight, %encoder.5.layers.5.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%664 : Float(1:118272, 512:231, 231:1) = onnx::Relu(%663) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%665 : Float(1:118272, 512:231, 231:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[1]](%664, %encoder.5.layers.9.0.weight, %encoder.5.layers.9.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%666 : Float(1:118272, 512:231, 231:1) = onnx::Relu(%665) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%667 : Float(1:512, 512:1) = onnx::ReduceMean[axes=[2], keepdims=0](%666) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%668 : Float(1:512, 512:1, 1:1) = onnx::Unsqueeze[axes=[2]](%667) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%669 : Float(1:102, 102:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%668, %encoder.5.layers.13._se_reduce.weight, %encoder.5.layers.13._se_reduce.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%670 : Float(1:102, 102:1, 1:1) = onnx::Sigmoid(%669) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%671 : Float(1:102, 102:1, 1:1) = onnx::Mul(%669, %670) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:28:0
%672 : Float(1:512, 512:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%671, %encoder.5.layers.13._se_expand.weight, %encoder.5.layers.13._se_expand.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%673 : Float(1:512, 512:1, 1:1) = onnx::Sigmoid(%672) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%674 : Float(1:118272, 512:231, 231:1) = onnx::Mul(%673, %666) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:59:0
%675 : Float(1:118272, 512:231, 231:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%674, %encoder.5.layers.14.0.weight, %encoder.5.layers.14.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%676 : Float(1:118272, 512:231, 231:1) = onnx::Relu(%675) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%677 : Float(1:118272, 512:231, 231:1) = onnx::Add(%676, %652) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:45:0
%678 : Float(1:118272, 512:231, 231:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[1]](%677, %encoder.6.layers.0.0.weight, %encoder.6.layers.0.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%679 : Float(1:118272, 512:231, 231:1) = onnx::Relu(%678) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%680 : Float(1:512, 512:1) = onnx::ReduceMean[axes=[2], keepdims=0](%679) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%681 : Float(1:512, 512:1, 1:1) = onnx::Unsqueeze[axes=[2]](%680) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%682 : Float(1:102, 102:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%681, %encoder.6.layers.4._se_reduce.weight, %encoder.6.layers.4._se_reduce.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%683 : Float(1:102, 102:1, 1:1) = onnx::Sigmoid(%682) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%684 : Float(1:102, 102:1, 1:1) = onnx::Mul(%682, %683) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:28:0
%685 : Float(1:512, 512:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%684, %encoder.6.layers.4._se_expand.weight, %encoder.6.layers.4._se_expand.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%686 : Float(1:512, 512:1, 1:1) = onnx::Sigmoid(%685) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%687 : Float(1:118272, 512:231, 231:1) = onnx::Mul(%686, %679) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:59:0
%688 : Float(1:118272, 512:231, 231:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%687, %encoder.6.layers.5.0.weight, %encoder.6.layers.5.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%689 : Float(1:118272, 512:231, 231:1) = onnx::Relu(%688) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%690 : Float(1:118272, 512:231, 231:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[1]](%689, %encoder.6.layers.9.0.weight, %encoder.6.layers.9.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%691 : Float(1:118272, 512:231, 231:1) = onnx::Relu(%690) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%692 : Float(1:512, 512:1) = onnx::ReduceMean[axes=[2], keepdims=0](%691) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%693 : Float(1:512, 512:1, 1:1) = onnx::Unsqueeze[axes=[2]](%692) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%694 : Float(1:102, 102:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%693, %encoder.6.layers.13._se_reduce.weight, %encoder.6.layers.13._se_reduce.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%695 : Float(1:102, 102:1, 1:1) = onnx::Sigmoid(%694) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%696 : Float(1:102, 102:1, 1:1) = onnx::Mul(%694, %695) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:28:0
%697 : Float(1:512, 512:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%696, %encoder.6.layers.13._se_expand.weight, %encoder.6.layers.13._se_expand.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%698 : Float(1:512, 512:1, 1:1) = onnx::Sigmoid(%697) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%699 : Float(1:118272, 512:231, 231:1) = onnx::Mul(%698, %691) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:59:0
%700 : Float(1:118272, 512:231, 231:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%699, %encoder.6.layers.14.0.weight, %encoder.6.layers.14.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%701 : Float(1:118272, 512:231, 231:1) = onnx::Relu(%700) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%702 : Float(1:118272, 512:231, 231:1) = onnx::Add(%701, %677) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:45:0
%703 : Float(1:59392, 512:116, 116:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[2]](%702, %encoder.7.layers.0.0.weight, %encoder.7.layers.0.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%704 : Float(1:59392, 512:116, 116:1) = onnx::Relu(%703) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%705 : Float(1:59392, 512:116, 116:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%704, %encoder.7.layers.4.0.weight, %encoder.7.layers.4.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%706 : Float(1:59392, 512:116, 116:1) = onnx::Relu(%705) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%707 : Float(1:59392, 512:116, 116:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[1]](%706, %encoder.8.layers.0.0.weight, %encoder.8.layers.0.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%708 : Float(1:59392, 512:116, 116:1) = onnx::Relu(%707) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%709 : Float(1:512, 512:1) = onnx::ReduceMean[axes=[2], keepdims=0](%708) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%710 : Float(1:512, 512:1, 1:1) = onnx::Unsqueeze[axes=[2]](%709) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%711 : Float(1:102, 102:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%710, %encoder.8.layers.4._se_reduce.weight, %encoder.8.layers.4._se_reduce.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%712 : Float(1:102, 102:1, 1:1) = onnx::Sigmoid(%711) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%713 : Float(1:102, 102:1, 1:1) = onnx::Mul(%711, %712) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:28:0
%714 : Float(1:512, 512:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%713, %encoder.8.layers.4._se_expand.weight, %encoder.8.layers.4._se_expand.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%715 : Float(1:512, 512:1, 1:1) = onnx::Sigmoid(%714) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%716 : Float(1:59392, 512:116, 116:1) = onnx::Mul(%715, %708) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:59:0
%717 : Float(1:59392, 512:116, 116:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%716, %encoder.8.layers.5.0.weight, %encoder.8.layers.5.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%718 : Float(1:59392, 512:116, 116:1) = onnx::Relu(%717) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%719 : Float(1:59392, 512:116, 116:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[1]](%718, %encoder.8.layers.9.0.weight, %encoder.8.layers.9.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%720 : Float(1:59392, 512:116, 116:1) = onnx::Relu(%719) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%721 : Float(1:512, 512:1) = onnx::ReduceMean[axes=[2], keepdims=0](%720) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%722 : Float(1:512, 512:1, 1:1) = onnx::Unsqueeze[axes=[2]](%721) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%723 : Float(1:102, 102:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%722, %encoder.8.layers.13._se_reduce.weight, %encoder.8.layers.13._se_reduce.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%724 : Float(1:102, 102:1, 1:1) = onnx::Sigmoid(%723) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%725 : Float(1:102, 102:1, 1:1) = onnx::Mul(%723, %724) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:28:0
%726 : Float(1:512, 512:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%725, %encoder.8.layers.13._se_expand.weight, %encoder.8.layers.13._se_expand.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%727 : Float(1:512, 512:1, 1:1) = onnx::Sigmoid(%726) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%728 : Float(1:59392, 512:116, 116:1) = onnx::Mul(%727, %720) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:59:0
%729 : Float(1:59392, 512:116, 116:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%728, %encoder.8.layers.14.0.weight, %encoder.8.layers.14.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%730 : Float(1:59392, 512:116, 116:1) = onnx::Relu(%729) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%731 : Float(1:59392, 512:116, 116:1) = onnx::Add(%730, %706) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:45:0
%732 : Float(1:59392, 512:116, 116:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[1]](%731, %encoder.9.layers.0.0.weight, %encoder.9.layers.0.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%733 : Float(1:59392, 512:116, 116:1) = onnx::Relu(%732) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%734 : Float(1:512, 512:1) = onnx::ReduceMean[axes=[2], keepdims=0](%733) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%735 : Float(1:512, 512:1, 1:1) = onnx::Unsqueeze[axes=[2]](%734) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%736 : Float(1:102, 102:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%735, %encoder.9.layers.4._se_reduce.weight, %encoder.9.layers.4._se_reduce.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%737 : Float(1:102, 102:1, 1:1) = onnx::Sigmoid(%736) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%738 : Float(1:102, 102:1, 1:1) = onnx::Mul(%736, %737) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:28:0
%739 : Float(1:512, 512:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%738, %encoder.9.layers.4._se_expand.weight, %encoder.9.layers.4._se_expand.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%740 : Float(1:512, 512:1, 1:1) = onnx::Sigmoid(%739) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%741 : Float(1:59392, 512:116, 116:1) = onnx::Mul(%740, %733) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:59:0
%742 : Float(1:59392, 512:116, 116:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%741, %encoder.9.layers.5.0.weight, %encoder.9.layers.5.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%743 : Float(1:59392, 512:116, 116:1) = onnx::Relu(%742) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%744 : Float(1:59392, 512:116, 116:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[1]](%743, %encoder.9.layers.9.0.weight, %encoder.9.layers.9.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%745 : Float(1:59392, 512:116, 116:1) = onnx::Relu(%744) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%746 : Float(1:512, 512:1) = onnx::ReduceMean[axes=[2], keepdims=0](%745) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%747 : Float(1:512, 512:1, 1:1) = onnx::Unsqueeze[axes=[2]](%746) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%748 : Float(1:102, 102:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%747, %encoder.9.layers.13._se_reduce.weight, %encoder.9.layers.13._se_reduce.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%749 : Float(1:102, 102:1, 1:1) = onnx::Sigmoid(%748) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%750 : Float(1:102, 102:1, 1:1) = onnx::Mul(%748, %749) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:28:0
%751 : Float(1:512, 512:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%750, %encoder.9.layers.13._se_expand.weight, %encoder.9.layers.13._se_expand.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%752 : Float(1:512, 512:1, 1:1) = onnx::Sigmoid(%751) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%753 : Float(1:59392, 512:116, 116:1) = onnx::Mul(%752, %745) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:59:0
%754 : Float(1:59392, 512:116, 116:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%753, %encoder.9.layers.14.0.weight, %encoder.9.layers.14.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%755 : Float(1:59392, 512:116, 116:1) = onnx::Relu(%754) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%756 : Float(1:59392, 512:116, 116:1) = onnx::Add(%755, %731) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:45:0
%757 : Float(1:59392, 512:116, 116:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[1]](%756, %encoder.10.layers.0.0.weight, %encoder.10.layers.0.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%758 : Float(1:59392, 512:116, 116:1) = onnx::Relu(%757) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%759 : Float(1:512, 512:1) = onnx::ReduceMean[axes=[2], keepdims=0](%758) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%760 : Float(1:512, 512:1, 1:1) = onnx::Unsqueeze[axes=[2]](%759) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%761 : Float(1:102, 102:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%760, %encoder.10.layers.4._se_reduce.weight, %encoder.10.layers.4._se_reduce.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%762 : Float(1:102, 102:1, 1:1) = onnx::Sigmoid(%761) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%763 : Float(1:102, 102:1, 1:1) = onnx::Mul(%761, %762) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:28:0
%764 : Float(1:512, 512:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%763, %encoder.10.layers.4._se_expand.weight, %encoder.10.layers.4._se_expand.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%765 : Float(1:512, 512:1, 1:1) = onnx::Sigmoid(%764) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%766 : Float(1:59392, 512:116, 116:1) = onnx::Mul(%765, %758) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:59:0
%767 : Float(1:59392, 512:116, 116:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%766, %encoder.10.layers.5.0.weight, %encoder.10.layers.5.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%768 : Float(1:59392, 512:116, 116:1) = onnx::Relu(%767) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%769 : Float(1:59392, 512:116, 116:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[1]](%768, %encoder.10.layers.9.0.weight, %encoder.10.layers.9.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%770 : Float(1:59392, 512:116, 116:1) = onnx::Relu(%769) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%771 : Float(1:512, 512:1) = onnx::ReduceMean[axes=[2], keepdims=0](%770) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%772 : Float(1:512, 512:1, 1:1) = onnx::Unsqueeze[axes=[2]](%771) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%773 : Float(1:102, 102:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%772, %encoder.10.layers.13._se_reduce.weight, %encoder.10.layers.13._se_reduce.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%774 : Float(1:102, 102:1, 1:1) = onnx::Sigmoid(%773) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%775 : Float(1:102, 102:1, 1:1) = onnx::Mul(%773, %774) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:28:0
%776 : Float(1:512, 512:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%775, %encoder.10.layers.13._se_expand.weight, %encoder.10.layers.13._se_expand.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%777 : Float(1:512, 512:1, 1:1) = onnx::Sigmoid(%776) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%778 : Float(1:59392, 512:116, 116:1) = onnx::Mul(%777, %770) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:59:0
%779 : Float(1:59392, 512:116, 116:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%778, %encoder.10.layers.14.0.weight, %encoder.10.layers.14.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%780 : Float(1:59392, 512:116, 116:1) = onnx::Relu(%779) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%781 : Float(1:59392, 512:116, 116:1) = onnx::Add(%780, %756) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:45:0
%782 : Float(1:59392, 512:116, 116:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[1]](%781, %encoder.11.layers.0.0.weight, %encoder.11.layers.0.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%783 : Float(1:59392, 512:116, 116:1) = onnx::Relu(%782) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%784 : Float(1:512, 512:1) = onnx::ReduceMean[axes=[2], keepdims=0](%783) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%785 : Float(1:512, 512:1, 1:1) = onnx::Unsqueeze[axes=[2]](%784) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%786 : Float(1:102, 102:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%785, %encoder.11.layers.4._se_reduce.weight, %encoder.11.layers.4._se_reduce.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%787 : Float(1:102, 102:1, 1:1) = onnx::Sigmoid(%786) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%788 : Float(1:102, 102:1, 1:1) = onnx::Mul(%786, %787) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:28:0
%789 : Float(1:512, 512:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%788, %encoder.11.layers.4._se_expand.weight, %encoder.11.layers.4._se_expand.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%790 : Float(1:512, 512:1, 1:1) = onnx::Sigmoid(%789) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%791 : Float(1:59392, 512:116, 116:1) = onnx::Mul(%790, %783) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:59:0
%792 : Float(1:59392, 512:116, 116:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%791, %encoder.11.layers.5.0.weight, %encoder.11.layers.5.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%793 : Float(1:59392, 512:116, 116:1) = onnx::Relu(%792) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%794 : Float(1:59392, 512:116, 116:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[1]](%793, %encoder.11.layers.9.0.weight, %encoder.11.layers.9.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%795 : Float(1:59392, 512:116, 116:1) = onnx::Relu(%794) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%796 : Float(1:512, 512:1) = onnx::ReduceMean[axes=[2], keepdims=0](%795) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%797 : Float(1:512, 512:1, 1:1) = onnx::Unsqueeze[axes=[2]](%796) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%798 : Float(1:102, 102:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%797, %encoder.11.layers.13._se_reduce.weight, %encoder.11.layers.13._se_reduce.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%799 : Float(1:102, 102:1, 1:1) = onnx::Sigmoid(%798) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%800 : Float(1:102, 102:1, 1:1) = onnx::Mul(%798, %799) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:28:0
%801 : Float(1:512, 512:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%800, %encoder.11.layers.13._se_expand.weight, %encoder.11.layers.13._se_expand.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%802 : Float(1:512, 512:1, 1:1) = onnx::Sigmoid(%801) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%803 : Float(1:59392, 512:116, 116:1) = onnx::Mul(%802, %795) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:59:0
%804 : Float(1:59392, 512:116, 116:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%803, %encoder.11.layers.14.0.weight, %encoder.11.layers.14.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%805 : Float(1:59392, 512:116, 116:1) = onnx::Relu(%804) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%806 : Float(1:59392, 512:116, 116:1) = onnx::Add(%805, %781) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:45:0
%807 : Float(1:59392, 512:116, 116:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[1]](%806, %encoder.12.layers.0.0.weight, %encoder.12.layers.0.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%808 : Float(1:59392, 512:116, 116:1) = onnx::Relu(%807) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%809 : Float(1:512, 512:1) = onnx::ReduceMean[axes=[2], keepdims=0](%808) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%810 : Float(1:512, 512:1, 1:1) = onnx::Unsqueeze[axes=[2]](%809) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%811 : Float(1:102, 102:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%810, %encoder.12.layers.4._se_reduce.weight, %encoder.12.layers.4._se_reduce.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%812 : Float(1:102, 102:1, 1:1) = onnx::Sigmoid(%811) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%813 : Float(1:102, 102:1, 1:1) = onnx::Mul(%811, %812) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:28:0
%814 : Float(1:512, 512:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%813, %encoder.12.layers.4._se_expand.weight, %encoder.12.layers.4._se_expand.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%815 : Float(1:512, 512:1, 1:1) = onnx::Sigmoid(%814) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%816 : Float(1:59392, 512:116, 116:1) = onnx::Mul(%815, %808) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:59:0
%817 : Float(1:59392, 512:116, 116:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%816, %encoder.12.layers.5.0.weight, %encoder.12.layers.5.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%818 : Float(1:59392, 512:116, 116:1) = onnx::Relu(%817) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%819 : Float(1:59392, 512:116, 116:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[1]](%818, %encoder.12.layers.9.0.weight, %encoder.12.layers.9.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%820 : Float(1:59392, 512:116, 116:1) = onnx::Relu(%819) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%821 : Float(1:512, 512:1) = onnx::ReduceMean[axes=[2], keepdims=0](%820) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%822 : Float(1:512, 512:1, 1:1) = onnx::Unsqueeze[axes=[2]](%821) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%823 : Float(1:102, 102:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%822, %encoder.12.layers.13._se_reduce.weight, %encoder.12.layers.13._se_reduce.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%824 : Float(1:102, 102:1, 1:1) = onnx::Sigmoid(%823) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%825 : Float(1:102, 102:1, 1:1) = onnx::Mul(%823, %824) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:28:0
%826 : Float(1:512, 512:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%825, %encoder.12.layers.13._se_expand.weight, %encoder.12.layers.13._se_expand.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%827 : Float(1:512, 512:1, 1:1) = onnx::Sigmoid(%826) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%828 : Float(1:59392, 512:116, 116:1) = onnx::Mul(%827, %820) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:59:0
%829 : Float(1:59392, 512:116, 116:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%828, %encoder.12.layers.14.0.weight, %encoder.12.layers.14.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%830 : Float(1:59392, 512:116, 116:1) = onnx::Relu(%829) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%831 : Float(1:59392, 512:116, 116:1) = onnx::Add(%830, %806) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:45:0
%832 : Float(1:59392, 512:116, 116:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[1]](%831, %encoder.13.layers.0.0.weight, %encoder.13.layers.0.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%833 : Float(1:59392, 512:116, 116:1) = onnx::Relu(%832) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%834 : Float(1:512, 512:1) = onnx::ReduceMean[axes=[2], keepdims=0](%833) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%835 : Float(1:512, 512:1, 1:1) = onnx::Unsqueeze[axes=[2]](%834) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%836 : Float(1:102, 102:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%835, %encoder.13.layers.4._se_reduce.weight, %encoder.13.layers.4._se_reduce.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%837 : Float(1:102, 102:1, 1:1) = onnx::Sigmoid(%836) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%838 : Float(1:102, 102:1, 1:1) = onnx::Mul(%836, %837) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:28:0
%839 : Float(1:512, 512:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%838, %encoder.13.layers.4._se_expand.weight, %encoder.13.layers.4._se_expand.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%840 : Float(1:512, 512:1, 1:1) = onnx::Sigmoid(%839) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%841 : Float(1:59392, 512:116, 116:1) = onnx::Mul(%840, %833) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:59:0
%842 : Float(1:59392, 512:116, 116:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%841, %encoder.13.layers.5.0.weight, %encoder.13.layers.5.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%843 : Float(1:59392, 512:116, 116:1) = onnx::Relu(%842) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%844 : Float(1:59392, 512:116, 116:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[1]](%843, %encoder.13.layers.9.0.weight, %encoder.13.layers.9.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%845 : Float(1:59392, 512:116, 116:1) = onnx::Relu(%844) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%846 : Float(1:512, 512:1) = onnx::ReduceMean[axes=[2], keepdims=0](%845) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%847 : Float(1:512, 512:1, 1:1) = onnx::Unsqueeze[axes=[2]](%846) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%848 : Float(1:102, 102:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%847, %encoder.13.layers.13._se_reduce.weight, %encoder.13.layers.13._se_reduce.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%849 : Float(1:102, 102:1, 1:1) = onnx::Sigmoid(%848) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%850 : Float(1:102, 102:1, 1:1) = onnx::Mul(%848, %849) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:28:0
%851 : Float(1:512, 512:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%850, %encoder.13.layers.13._se_expand.weight, %encoder.13.layers.13._se_expand.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%852 : Float(1:512, 512:1, 1:1) = onnx::Sigmoid(%851) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%853 : Float(1:59392, 512:116, 116:1) = onnx::Mul(%852, %845) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:59:0
%854 : Float(1:59392, 512:116, 116:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%853, %encoder.13.layers.14.0.weight, %encoder.13.layers.14.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%855 : Float(1:59392, 512:116, 116:1) = onnx::Relu(%854) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%856 : Float(1:59392, 512:116, 116:1) = onnx::Add(%855, %831) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:45:0
%857 : Float(1:29696, 512:58, 58:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[2]](%856, %encoder.14.layers.0.0.weight, %encoder.14.layers.0.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%858 : Float(1:29696, 512:58, 58:1) = onnx::Relu(%857) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%859 : Float(1:29696, 512:58, 58:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%858, %encoder.14.layers.4.0.weight, %encoder.14.layers.4.0.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%860 : Float(1:29696, 512:58, 58:1) = onnx::Relu(%859) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%861 : Float(1:29696, 512:58, 58:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[1]](%860, %encoder.15.layers.0.weight) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%862 : Float(1:29696, 512:58, 58:1) = onnx::BatchNormalization[epsilon=1.0000000000000001e-05, momentum=0.90000000000000002](%861, %encoder.15.layers.1.weight, %encoder.15.layers.1.bias, %encoder.15.layers.1.running_mean, %encoder.15.layers.1.running_var) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2016:0
%863 : Float(1:29696, 512:58, 58:1) = onnx::Relu(%862) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%864 : Float(1:512, 512:1) = onnx::ReduceMean[axes=[2], keepdims=0](%863) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%865 : Float(1:512, 512:1, 1:1) = onnx::Unsqueeze[axes=[2]](%864) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%866 : Float(1:102, 102:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%865, %encoder.15.layers.4._se_reduce.weight, %encoder.15.layers.4._se_reduce.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%867 : Float(1:102, 102:1, 1:1) = onnx::Sigmoid(%866) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%868 : Float(1:102, 102:1, 1:1) = onnx::Mul(%866, %867) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:28:0
%869 : Float(1:512, 512:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%868, %encoder.15.layers.4._se_expand.weight, %encoder.15.layers.4._se_expand.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%870 : Float(1:512, 512:1, 1:1) = onnx::Sigmoid(%869) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%871 : Float(1:29696, 512:58, 58:1) = onnx::Mul(%870, %863) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:59:0
%872 : Float(1:29696, 512:58, 58:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%871, %encoder.15.layers.5.weight, %encoder.15.layers.5.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%873 : Float(1:29696, 512:58, 58:1) = onnx::BatchNormalization[epsilon=1.0000000000000001e-05, momentum=0.90000000000000002](%872, %encoder.15.layers.6.weight, %encoder.15.layers.6.bias, %encoder.15.layers.6.running_mean, %encoder.15.layers.6.running_var) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2016:0
%874 : Float(1:29696, 512:58, 58:1) = onnx::Relu(%873) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%875 : Float(1:29696, 512:58, 58:1) = onnx::Add(%874, %860) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:45:0
%876 : Float(1:29696, 512:58, 58:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[1]](%875, %encoder.16.layers.0.weight) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%877 : Float(1:29696, 512:58, 58:1) = onnx::BatchNormalization[epsilon=1.0000000000000001e-05, momentum=0.90000000000000002](%876, %encoder.16.layers.1.weight, %encoder.16.layers.1.bias, %encoder.16.layers.1.running_mean, %encoder.16.layers.1.running_var) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2016:0
%878 : Float(1:29696, 512:58, 58:1) = onnx::Relu(%877) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%879 : Float(1:512, 512:1) = onnx::ReduceMean[axes=[2], keepdims=0](%878) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%880 : Float(1:512, 512:1, 1:1) = onnx::Unsqueeze[axes=[2]](%879) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%881 : Float(1:102, 102:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%880, %encoder.16.layers.4._se_reduce.weight, %encoder.16.layers.4._se_reduce.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%882 : Float(1:102, 102:1, 1:1) = onnx::Sigmoid(%881) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%883 : Float(1:102, 102:1, 1:1) = onnx::Mul(%881, %882) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:28:0
%884 : Float(1:512, 512:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%883, %encoder.16.layers.4._se_expand.weight, %encoder.16.layers.4._se_expand.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%885 : Float(1:512, 512:1, 1:1) = onnx::Sigmoid(%884) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%886 : Float(1:29696, 512:58, 58:1) = onnx::Mul(%885, %878) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:59:0
%887 : Float(1:29696, 512:58, 58:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%886, %encoder.16.layers.5.weight, %encoder.16.layers.5.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%888 : Float(1:29696, 512:58, 58:1) = onnx::BatchNormalization[epsilon=1.0000000000000001e-05, momentum=0.90000000000000002](%887, %encoder.16.layers.6.weight, %encoder.16.layers.6.bias, %encoder.16.layers.6.running_mean, %encoder.16.layers.6.running_var) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2016:0
%889 : Float(1:29696, 512:58, 58:1) = onnx::Relu(%888) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%890 : Float(1:29696, 512:58, 58:1) = onnx::Add(%889, %875) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:45:0
%891 : Float(1:29696, 512:58, 58:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[1]](%890, %encoder.17.layers.0.weight) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%892 : Float(1:29696, 512:58, 58:1) = onnx::BatchNormalization[epsilon=1.0000000000000001e-05, momentum=0.90000000000000002](%891, %encoder.17.layers.1.weight, %encoder.17.layers.1.bias, %encoder.17.layers.1.running_mean, %encoder.17.layers.1.running_var) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2016:0
%893 : Float(1:29696, 512:58, 58:1) = onnx::Relu(%892) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%894 : Float(1:512, 512:1) = onnx::ReduceMean[axes=[2], keepdims=0](%893) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%895 : Float(1:512, 512:1, 1:1) = onnx::Unsqueeze[axes=[2]](%894) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%896 : Float(1:102, 102:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%895, %encoder.17.layers.4._se_reduce.weight, %encoder.17.layers.4._se_reduce.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%897 : Float(1:102, 102:1, 1:1) = onnx::Sigmoid(%896) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%898 : Float(1:102, 102:1, 1:1) = onnx::Mul(%896, %897) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:28:0
%899 : Float(1:512, 512:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%898, %encoder.17.layers.4._se_expand.weight, %encoder.17.layers.4._se_expand.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%900 : Float(1:512, 512:1, 1:1) = onnx::Sigmoid(%899) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%901 : Float(1:29696, 512:58, 58:1) = onnx::Mul(%900, %893) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:59:0
%902 : Float(1:29696, 512:58, 58:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%901, %encoder.17.layers.5.weight, %encoder.17.layers.5.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%903 : Float(1:29696, 512:58, 58:1) = onnx::BatchNormalization[epsilon=1.0000000000000001e-05, momentum=0.90000000000000002](%902, %encoder.17.layers.6.weight, %encoder.17.layers.6.bias, %encoder.17.layers.6.running_mean, %encoder.17.layers.6.running_var) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2016:0
%904 : Float(1:29696, 512:58, 58:1) = onnx::Relu(%903) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%905 : Float(1:29696, 512:58, 58:1) = onnx::Add(%904, %890) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:45:0
%906 : Float(1:29696, 512:58, 58:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[1]](%905, %encoder.18.layers.0.weight) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%907 : Float(1:29696, 512:58, 58:1) = onnx::BatchNormalization[epsilon=1.0000000000000001e-05, momentum=0.90000000000000002](%906, %encoder.18.layers.1.weight, %encoder.18.layers.1.bias, %encoder.18.layers.1.running_mean, %encoder.18.layers.1.running_var) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2016:0
%908 : Float(1:29696, 512:58, 58:1) = onnx::Relu(%907) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%909 : Float(1:512, 512:1) = onnx::ReduceMean[axes=[2], keepdims=0](%908) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%910 : Float(1:512, 512:1, 1:1) = onnx::Unsqueeze[axes=[2]](%909) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%911 : Float(1:102, 102:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%910, %encoder.18.layers.4._se_reduce.weight, %encoder.18.layers.4._se_reduce.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%912 : Float(1:102, 102:1, 1:1) = onnx::Sigmoid(%911) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%913 : Float(1:102, 102:1, 1:1) = onnx::Mul(%911, %912) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:28:0
%914 : Float(1:512, 512:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%913, %encoder.18.layers.4._se_expand.weight, %encoder.18.layers.4._se_expand.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%915 : Float(1:512, 512:1, 1:1) = onnx::Sigmoid(%914) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%916 : Float(1:29696, 512:58, 58:1) = onnx::Mul(%915, %908) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:59:0
%917 : Float(1:29696, 512:58, 58:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%916, %encoder.18.layers.5.weight, %encoder.18.layers.5.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%918 : Float(1:29696, 512:58, 58:1) = onnx::BatchNormalization[epsilon=1.0000000000000001e-05, momentum=0.90000000000000002](%917, %encoder.18.layers.6.weight, %encoder.18.layers.6.bias, %encoder.18.layers.6.running_mean, %encoder.18.layers.6.running_var) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2016:0
%919 : Float(1:29696, 512:58, 58:1) = onnx::Relu(%918) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%920 : Float(1:29696, 512:58, 58:1) = onnx::Add(%919, %905) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:45:0
%921 : Float(1:29696, 512:58, 58:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[1]](%920, %encoder.19.layers.0.weight) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%922 : Float(1:29696, 512:58, 58:1) = onnx::BatchNormalization[epsilon=1.0000000000000001e-05, momentum=0.90000000000000002](%921, %encoder.19.layers.1.weight, %encoder.19.layers.1.bias, %encoder.19.layers.1.running_mean, %encoder.19.layers.1.running_var) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2016:0
%923 : Float(1:29696, 512:58, 58:1) = onnx::Relu(%922) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%924 : Float(1:512, 512:1) = onnx::ReduceMean[axes=[2], keepdims=0](%923) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%925 : Float(1:512, 512:1, 1:1) = onnx::Unsqueeze[axes=[2]](%924) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%926 : Float(1:102, 102:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%925, %encoder.19.layers.4._se_reduce.weight, %encoder.19.layers.4._se_reduce.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%927 : Float(1:102, 102:1, 1:1) = onnx::Sigmoid(%926) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%928 : Float(1:102, 102:1, 1:1) = onnx::Mul(%926, %927) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:28:0
%929 : Float(1:512, 512:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%928, %encoder.19.layers.4._se_expand.weight, %encoder.19.layers.4._se_expand.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%930 : Float(1:512, 512:1, 1:1) = onnx::Sigmoid(%929) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%931 : Float(1:29696, 512:58, 58:1) = onnx::Mul(%930, %923) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:59:0
%932 : Float(1:29696, 512:58, 58:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%931, %encoder.19.layers.5.weight, %encoder.19.layers.5.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%933 : Float(1:29696, 512:58, 58:1) = onnx::BatchNormalization[epsilon=1.0000000000000001e-05, momentum=0.90000000000000002](%932, %encoder.19.layers.6.weight, %encoder.19.layers.6.bias, %encoder.19.layers.6.running_mean, %encoder.19.layers.6.running_var) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2016:0
%934 : Float(1:29696, 512:58, 58:1) = onnx::Relu(%933) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%935 : Float(1:29696, 512:58, 58:1) = onnx::Add(%934, %920) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:45:0
%936 : Float(1:29696, 512:58, 58:1) = onnx::Conv[dilations=[1], group=8, kernel_shape=[7], pads=[3, 3], strides=[1]](%935, %encoder.20.layers.0.weight) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%937 : Float(1:29696, 512:58, 58:1) = onnx::BatchNormalization[epsilon=1.0000000000000001e-05, momentum=0.90000000000000002](%936, %encoder.20.layers.1.weight, %encoder.20.layers.1.bias, %encoder.20.layers.1.running_mean, %encoder.20.layers.1.running_var) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2016:0
%938 : Float(1:29696, 512:58, 58:1) = onnx::Relu(%937) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%939 : Float(1:512, 512:1) = onnx::ReduceMean[axes=[2], keepdims=0](%938) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%940 : Float(1:512, 512:1, 1:1) = onnx::Unsqueeze[axes=[2]](%939) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:69:0
%941 : Float(1:102, 102:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%940, %encoder.20.layers.4._se_reduce.weight, %encoder.20.layers.4._se_reduce.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%942 : Float(1:102, 102:1, 1:1) = onnx::Sigmoid(%941) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%943 : Float(1:102, 102:1, 1:1) = onnx::Mul(%941, %942) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model_blocks.py:28:0
%944 : Float(1:512, 512:1, 1:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%943, %encoder.20.layers.4._se_expand.weight, %encoder.20.layers.4._se_expand.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%945 : Float(1:512, 512:1, 1:1) = onnx::Sigmoid(%944) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/activation.py:299:0
%946 : Float(1:29696, 512:58, 58:1) = onnx::Mul(%945, %938) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:59:0
%947 : Float(1:29696, 512:58, 58:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%946, %encoder.20.layers.5.weight, %encoder.20.layers.5.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%948 : Float(1:29696, 512:58, 58:1) = onnx::BatchNormalization[epsilon=1.0000000000000001e-05, momentum=0.90000000000000002](%947, %encoder.20.layers.6.weight, %encoder.20.layers.6.bias, %encoder.20.layers.6.running_mean, %encoder.20.layers.6.running_var) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2016:0
%949 : Float(1:29696, 512:58, 58:1) = onnx::Relu(%948) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%950 : Float(1:29696, 512:58, 58:1) = onnx::Add(%949, %935) # /opt/conda/lib/python3.7/site-packages/torch/nn/quantized/modules/functional_modules.py:45:0
%951 : Float(58:512, 1:512, 512:1) = onnx::Transpose[perm=[2, 0, 1]](%950) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model.py:668:0
%952 : Tensor = onnx::Shape(%951)
%953 : Tensor = onnx::Constant[value={0}]()
%954 : Long() = onnx::Gather[axis=0](%952, %953) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%955 : Tensor = onnx::Shape(%951)
%956 : Tensor = onnx::Constant[value={1}]()
%957 : Long() = onnx::Gather[axis=0](%955, %956) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%958 : Tensor = onnx::Shape(%951)
%959 : Tensor = onnx::Constant[value={2}]()
%960 : Long() = onnx::Gather[axis=0](%958, %959) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%961 : Long() = onnx::Constant[value={2}]()
%962 : LongTensor = onnx::Div(%960, %961)
%963 : Tensor = onnx::Cast[to=7](%962)
%964 : Long() = onnx::Cast[to=7](%963) # /opt/conda/lib/python3.7/site-packages/torch/tensor.py:424:0
%966 : Float(58:1536, 1:1536, 1536:1) = onnx::MatMul(%951, %2463) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%967 : Float(58:1536, 1:1536, 1536:1) = onnx::Add(%966, %decoder.layers.0.self_attn.in_proj_bias)
%968 : Float(58:1536, 1:1536, 512:1), %969 : Float(58:512, 1:512, 512:1), %970 : Float(58:512, 1:512, 512:1) = onnx::Split[axis=-1, split=[512, 512, 512]](%967) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%971 : Float() = onnx::Constant[value={0.0625}]()
%972 : Float(58:512, 1:512, 512:1) = onnx::Mul(%968, %971) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%973 : Long() = onnx::Constant[value={2}]()
%974 : Long() = onnx::Mul(%957, %973)
%975 : Tensor = onnx::Unsqueeze[axes=[0]](%954)
%976 : Tensor = onnx::Unsqueeze[axes=[0]](%974)
%977 : Tensor = onnx::Unsqueeze[axes=[0]](%964)
%978 : Tensor = onnx::Concat[axis=0](%975, %976, %977)
%979 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%972, %978) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%980 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%979) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%981 : Long() = onnx::Constant[value={2}]()
%982 : Long() = onnx::Mul(%957, %981)
%985 : Tensor = onnx::Unsqueeze[axes=[0]](%982)
%986 : Tensor = onnx::Unsqueeze[axes=[0]](%964)
%987 : Tensor = onnx::Concat[axis=0](%2464, %985, %986)
%988 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%969, %987) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4079:0
%989 : Long() = onnx::Constant[value={2}]()
%990 : Long() = onnx::Mul(%957, %989)
%993 : Tensor = onnx::Unsqueeze[axes=[0]](%990)
%994 : Tensor = onnx::Unsqueeze[axes=[0]](%964)
%995 : Tensor = onnx::Concat[axis=0](%2465, %993, %994)
%996 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%970, %995) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%997 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%996) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%998 : Float(2:256, 256:1, 58:512) = onnx::Transpose[perm=[1, 2, 0]](%988) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%999 : Float(2:3364, 58:58, 58:1) = onnx::MatMul(%980, %998) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%1000 : Float(2:3364, 58:58, 58:1) = onnx::Softmax[axis=2](%999) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1001 : Float(2:14848, 58:256, 256:1) = onnx::MatMul(%1000, %997) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4130:0
%1002 : Float(58:512, 2:256, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1001) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%1003 : Tensor = onnx::Unsqueeze[axes=[0]](%954)
%1004 : Tensor = onnx::Unsqueeze[axes=[0]](%957)
%1005 : Tensor = onnx::Unsqueeze[axes=[0]](%960)
%1006 : Tensor = onnx::Concat[axis=0](%1003, %1004, %1005)
%1007 : Float(58:512, 1:512, 512:1) = onnx::Reshape(%1002, %1006) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%1009 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1007, %2466) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1010 : Float(58:512, 1:512, 512:1) = onnx::Add(%1009, %decoder.layers.0.self_attn.out_proj.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1011 : Float(58:512, 1:512, 512:1) = onnx::Add(%951, %1010) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:295:0
%1013 : Tensor = onnx::ReduceMean[axes=[-1]](%1011)
%1014 : FloatTensor = onnx::Sub(%1011, %1013)
%1015 : Tensor = onnx::Cast[to=1](%1014)
%1017 : Tensor = onnx::Pow(%1015, %2467)
%1018 : Tensor = onnx::ReduceMean[axes=[-1]](%1017)
%1019 : Float() = onnx::Constant[value={1e-05}]()
%1020 : FloatTensor = onnx::Add(%1018, %1019)
%1021 : Tensor = onnx::Sqrt(%1020)
%1022 : FloatTensor = onnx::Div(%1014, %1021)
%1023 : FloatTensor = onnx::Mul(%1022, %decoder.layers.0.norm1.weight)
%1024 : Float(58:512, 1:512, 512:1) = onnx::Add(%1023, %decoder.layers.0.norm1.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%1026 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1024, %2468) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1027 : Float(58:512, 1:512, 512:1) = onnx::Add(%1026, %decoder.layers.0.linear1.bias)
%1028 : Float(58:512, 1:512, 512:1) = onnx::Relu(%1027) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1030 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1028, %2469) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1031 : Float(58:512, 1:512, 512:1) = onnx::Add(%1030, %decoder.layers.0.linear2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1032 : Float(58:512, 1:512, 512:1) = onnx::Add(%1024, %1031) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:298:0
%1034 : Tensor = onnx::ReduceMean[axes=[-1]](%1032)
%1035 : FloatTensor = onnx::Sub(%1032, %1034)
%1036 : Tensor = onnx::Cast[to=1](%1035)
%1038 : Tensor = onnx::Pow(%1036, %2470)
%1039 : Tensor = onnx::ReduceMean[axes=[-1]](%1038)
%1040 : Float() = onnx::Constant[value={1e-05}]()
%1041 : FloatTensor = onnx::Add(%1039, %1040)
%1042 : Tensor = onnx::Sqrt(%1041)
%1043 : FloatTensor = onnx::Div(%1035, %1042)
%1044 : FloatTensor = onnx::Mul(%1043, %decoder.layers.0.norm2.weight)
%1045 : Float(58:512, 1:512, 512:1) = onnx::Add(%1044, %decoder.layers.0.norm2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%1046 : Tensor = onnx::Shape(%1045)
%1047 : Tensor = onnx::Constant[value={0}]()
%1048 : Long() = onnx::Gather[axis=0](%1046, %1047) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1049 : Tensor = onnx::Shape(%1045)
%1050 : Tensor = onnx::Constant[value={1}]()
%1051 : Long() = onnx::Gather[axis=0](%1049, %1050) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1052 : Tensor = onnx::Shape(%1045)
%1053 : Tensor = onnx::Constant[value={2}]()
%1054 : Long() = onnx::Gather[axis=0](%1052, %1053) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1055 : Long() = onnx::Constant[value={2}]()
%1056 : LongTensor = onnx::Div(%1054, %1055)
%1057 : Tensor = onnx::Cast[to=7](%1056)
%1058 : Long() = onnx::Cast[to=7](%1057) # /opt/conda/lib/python3.7/site-packages/torch/tensor.py:424:0
%1060 : Float(58:1536, 1:1536, 1536:1) = onnx::MatMul(%1045, %2471) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1061 : Float(58:1536, 1:1536, 1536:1) = onnx::Add(%1060, %decoder.layers.1.self_attn.in_proj_bias)
%1062 : Float(58:1536, 1:1536, 512:1), %1063 : Float(58:512, 1:512, 512:1), %1064 : Float(58:512, 1:512, 512:1) = onnx::Split[axis=-1, split=[512, 512, 512]](%1061) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%1065 : Float() = onnx::Constant[value={0.0625}]()
%1066 : Float(58:512, 1:512, 512:1) = onnx::Mul(%1062, %1065) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%1067 : Long() = onnx::Constant[value={2}]()
%1068 : Long() = onnx::Mul(%1051, %1067)
%1069 : Tensor = onnx::Unsqueeze[axes=[0]](%1048)
%1070 : Tensor = onnx::Unsqueeze[axes=[0]](%1068)
%1071 : Tensor = onnx::Unsqueeze[axes=[0]](%1058)
%1072 : Tensor = onnx::Concat[axis=0](%1069, %1070, %1071)
%1073 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%1066, %1072) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%1074 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1073) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%1075 : Long() = onnx::Constant[value={2}]()
%1076 : Long() = onnx::Mul(%1051, %1075)
%1079 : Tensor = onnx::Unsqueeze[axes=[0]](%1076)
%1080 : Tensor = onnx::Unsqueeze[axes=[0]](%1058)
%1081 : Tensor = onnx::Concat[axis=0](%2472, %1079, %1080)
%1082 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%1063, %1081) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4079:0
%1083 : Long() = onnx::Constant[value={2}]()
%1084 : Long() = onnx::Mul(%1051, %1083)
%1087 : Tensor = onnx::Unsqueeze[axes=[0]](%1084)
%1088 : Tensor = onnx::Unsqueeze[axes=[0]](%1058)
%1089 : Tensor = onnx::Concat[axis=0](%2473, %1087, %1088)
%1090 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%1064, %1089) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%1091 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1090) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%1092 : Float(2:256, 256:1, 58:512) = onnx::Transpose[perm=[1, 2, 0]](%1082) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%1093 : Float(2:3364, 58:58, 58:1) = onnx::MatMul(%1074, %1092) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%1094 : Float(2:3364, 58:58, 58:1) = onnx::Softmax[axis=2](%1093) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1095 : Float(2:14848, 58:256, 256:1) = onnx::MatMul(%1094, %1091) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4130:0
%1096 : Float(58:512, 2:256, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1095) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%1097 : Tensor = onnx::Unsqueeze[axes=[0]](%1048)
%1098 : Tensor = onnx::Unsqueeze[axes=[0]](%1051)
%1099 : Tensor = onnx::Unsqueeze[axes=[0]](%1054)
%1100 : Tensor = onnx::Concat[axis=0](%1097, %1098, %1099)
%1101 : Float(58:512, 1:512, 512:1) = onnx::Reshape(%1096, %1100) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%1103 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1101, %2474) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1104 : Float(58:512, 1:512, 512:1) = onnx::Add(%1103, %decoder.layers.1.self_attn.out_proj.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1105 : Float(58:512, 1:512, 512:1) = onnx::Add(%1045, %1104) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:295:0
%1107 : Tensor = onnx::ReduceMean[axes=[-1]](%1105)
%1108 : FloatTensor = onnx::Sub(%1105, %1107)
%1109 : Tensor = onnx::Cast[to=1](%1108)
%1111 : Tensor = onnx::Pow(%1109, %2475)
%1112 : Tensor = onnx::ReduceMean[axes=[-1]](%1111)
%1113 : Float() = onnx::Constant[value={1e-05}]()
%1114 : FloatTensor = onnx::Add(%1112, %1113)
%1115 : Tensor = onnx::Sqrt(%1114)
%1116 : FloatTensor = onnx::Div(%1108, %1115)
%1117 : FloatTensor = onnx::Mul(%1116, %decoder.layers.1.norm1.weight)
%1118 : Float(58:512, 1:512, 512:1) = onnx::Add(%1117, %decoder.layers.1.norm1.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%1120 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1118, %2476) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1121 : Float(58:512, 1:512, 512:1) = onnx::Add(%1120, %decoder.layers.1.linear1.bias)
%1122 : Float(58:512, 1:512, 512:1) = onnx::Relu(%1121) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1124 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1122, %2477) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1125 : Float(58:512, 1:512, 512:1) = onnx::Add(%1124, %decoder.layers.1.linear2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1126 : Float(58:512, 1:512, 512:1) = onnx::Add(%1118, %1125) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:298:0
%1128 : Tensor = onnx::ReduceMean[axes=[-1]](%1126)
%1129 : FloatTensor = onnx::Sub(%1126, %1128)
%1130 : Tensor = onnx::Cast[to=1](%1129)
%1132 : Tensor = onnx::Pow(%1130, %2478)
%1133 : Tensor = onnx::ReduceMean[axes=[-1]](%1132)
%1134 : Float() = onnx::Constant[value={1e-05}]()
%1135 : FloatTensor = onnx::Add(%1133, %1134)
%1136 : Tensor = onnx::Sqrt(%1135)
%1137 : FloatTensor = onnx::Div(%1129, %1136)
%1138 : FloatTensor = onnx::Mul(%1137, %decoder.layers.1.norm2.weight)
%1139 : Float(58:512, 1:512, 512:1) = onnx::Add(%1138, %decoder.layers.1.norm2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%1140 : Tensor = onnx::Shape(%1139)
%1141 : Tensor = onnx::Constant[value={0}]()
%1142 : Long() = onnx::Gather[axis=0](%1140, %1141) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1143 : Tensor = onnx::Shape(%1139)
%1144 : Tensor = onnx::Constant[value={1}]()
%1145 : Long() = onnx::Gather[axis=0](%1143, %1144) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1146 : Tensor = onnx::Shape(%1139)
%1147 : Tensor = onnx::Constant[value={2}]()
%1148 : Long() = onnx::Gather[axis=0](%1146, %1147) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1149 : Long() = onnx::Constant[value={2}]()
%1150 : LongTensor = onnx::Div(%1148, %1149)
%1151 : Tensor = onnx::Cast[to=7](%1150)
%1152 : Long() = onnx::Cast[to=7](%1151) # /opt/conda/lib/python3.7/site-packages/torch/tensor.py:424:0
%1154 : Float(58:1536, 1:1536, 1536:1) = onnx::MatMul(%1139, %2479) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1155 : Float(58:1536, 1:1536, 1536:1) = onnx::Add(%1154, %decoder.layers.2.self_attn.in_proj_bias)
%1156 : Float(58:1536, 1:1536, 512:1), %1157 : Float(58:512, 1:512, 512:1), %1158 : Float(58:512, 1:512, 512:1) = onnx::Split[axis=-1, split=[512, 512, 512]](%1155) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%1159 : Float() = onnx::Constant[value={0.0625}]()
%1160 : Float(58:512, 1:512, 512:1) = onnx::Mul(%1156, %1159) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%1161 : Long() = onnx::Constant[value={2}]()
%1162 : Long() = onnx::Mul(%1145, %1161)
%1163 : Tensor = onnx::Unsqueeze[axes=[0]](%1142)
%1164 : Tensor = onnx::Unsqueeze[axes=[0]](%1162)
%1165 : Tensor = onnx::Unsqueeze[axes=[0]](%1152)
%1166 : Tensor = onnx::Concat[axis=0](%1163, %1164, %1165)
%1167 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%1160, %1166) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%1168 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1167) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%1169 : Long() = onnx::Constant[value={2}]()
%1170 : Long() = onnx::Mul(%1145, %1169)
%1173 : Tensor = onnx::Unsqueeze[axes=[0]](%1170)
%1174 : Tensor = onnx::Unsqueeze[axes=[0]](%1152)
%1175 : Tensor = onnx::Concat[axis=0](%2480, %1173, %1174)
%1176 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%1157, %1175) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4079:0
%1177 : Long() = onnx::Constant[value={2}]()
%1178 : Long() = onnx::Mul(%1145, %1177)
%1181 : Tensor = onnx::Unsqueeze[axes=[0]](%1178)
%1182 : Tensor = onnx::Unsqueeze[axes=[0]](%1152)
%1183 : Tensor = onnx::Concat[axis=0](%2481, %1181, %1182)
%1184 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%1158, %1183) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%1185 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1184) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%1186 : Float(2:256, 256:1, 58:512) = onnx::Transpose[perm=[1, 2, 0]](%1176) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%1187 : Float(2:3364, 58:58, 58:1) = onnx::MatMul(%1168, %1186) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%1188 : Float(2:3364, 58:58, 58:1) = onnx::Softmax[axis=2](%1187) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1189 : Float(2:14848, 58:256, 256:1) = onnx::MatMul(%1188, %1185) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4130:0
%1190 : Float(58:512, 2:256, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1189) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%1191 : Tensor = onnx::Unsqueeze[axes=[0]](%1142)
%1192 : Tensor = onnx::Unsqueeze[axes=[0]](%1145)
%1193 : Tensor = onnx::Unsqueeze[axes=[0]](%1148)
%1194 : Tensor = onnx::Concat[axis=0](%1191, %1192, %1193)
%1195 : Float(58:512, 1:512, 512:1) = onnx::Reshape(%1190, %1194) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%1197 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1195, %2482) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1198 : Float(58:512, 1:512, 512:1) = onnx::Add(%1197, %decoder.layers.2.self_attn.out_proj.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1199 : Float(58:512, 1:512, 512:1) = onnx::Add(%1139, %1198) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:295:0
%1201 : Tensor = onnx::ReduceMean[axes=[-1]](%1199)
%1202 : FloatTensor = onnx::Sub(%1199, %1201)
%1203 : Tensor = onnx::Cast[to=1](%1202)
%1205 : Tensor = onnx::Pow(%1203, %2483)
%1206 : Tensor = onnx::ReduceMean[axes=[-1]](%1205)
%1207 : Float() = onnx::Constant[value={1e-05}]()
%1208 : FloatTensor = onnx::Add(%1206, %1207)
%1209 : Tensor = onnx::Sqrt(%1208)
%1210 : FloatTensor = onnx::Div(%1202, %1209)
%1211 : FloatTensor = onnx::Mul(%1210, %decoder.layers.2.norm1.weight)
%1212 : Float(58:512, 1:512, 512:1) = onnx::Add(%1211, %decoder.layers.2.norm1.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%1214 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1212, %2484) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1215 : Float(58:512, 1:512, 512:1) = onnx::Add(%1214, %decoder.layers.2.linear1.bias)
%1216 : Float(58:512, 1:512, 512:1) = onnx::Relu(%1215) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1218 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1216, %2485) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1219 : Float(58:512, 1:512, 512:1) = onnx::Add(%1218, %decoder.layers.2.linear2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1220 : Float(58:512, 1:512, 512:1) = onnx::Add(%1212, %1219) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:298:0
%1222 : Tensor = onnx::ReduceMean[axes=[-1]](%1220)
%1223 : FloatTensor = onnx::Sub(%1220, %1222)
%1224 : Tensor = onnx::Cast[to=1](%1223)
%1226 : Tensor = onnx::Pow(%1224, %2486)
%1227 : Tensor = onnx::ReduceMean[axes=[-1]](%1226)
%1228 : Float() = onnx::Constant[value={1e-05}]()
%1229 : FloatTensor = onnx::Add(%1227, %1228)
%1230 : Tensor = onnx::Sqrt(%1229)
%1231 : FloatTensor = onnx::Div(%1223, %1230)
%1232 : FloatTensor = onnx::Mul(%1231, %decoder.layers.2.norm2.weight)
%1233 : Float(58:512, 1:512, 512:1) = onnx::Add(%1232, %decoder.layers.2.norm2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%1234 : Tensor = onnx::Shape(%1233)
%1235 : Tensor = onnx::Constant[value={0}]()
%1236 : Long() = onnx::Gather[axis=0](%1234, %1235) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1237 : Tensor = onnx::Shape(%1233)
%1238 : Tensor = onnx::Constant[value={1}]()
%1239 : Long() = onnx::Gather[axis=0](%1237, %1238) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1240 : Tensor = onnx::Shape(%1233)
%1241 : Tensor = onnx::Constant[value={2}]()
%1242 : Long() = onnx::Gather[axis=0](%1240, %1241) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1243 : Long() = onnx::Constant[value={2}]()
%1244 : LongTensor = onnx::Div(%1242, %1243)
%1245 : Tensor = onnx::Cast[to=7](%1244)
%1246 : Long() = onnx::Cast[to=7](%1245) # /opt/conda/lib/python3.7/site-packages/torch/tensor.py:424:0
%1248 : Float(58:1536, 1:1536, 1536:1) = onnx::MatMul(%1233, %2487) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1249 : Float(58:1536, 1:1536, 1536:1) = onnx::Add(%1248, %decoder.layers.3.self_attn.in_proj_bias)
%1250 : Float(58:1536, 1:1536, 512:1), %1251 : Float(58:512, 1:512, 512:1), %1252 : Float(58:512, 1:512, 512:1) = onnx::Split[axis=-1, split=[512, 512, 512]](%1249) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%1253 : Float() = onnx::Constant[value={0.0625}]()
%1254 : Float(58:512, 1:512, 512:1) = onnx::Mul(%1250, %1253) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%1255 : Long() = onnx::Constant[value={2}]()
%1256 : Long() = onnx::Mul(%1239, %1255)
%1257 : Tensor = onnx::Unsqueeze[axes=[0]](%1236)
%1258 : Tensor = onnx::Unsqueeze[axes=[0]](%1256)
%1259 : Tensor = onnx::Unsqueeze[axes=[0]](%1246)
%1260 : Tensor = onnx::Concat[axis=0](%1257, %1258, %1259)
%1261 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%1254, %1260) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%1262 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1261) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%1263 : Long() = onnx::Constant[value={2}]()
%1264 : Long() = onnx::Mul(%1239, %1263)
%1267 : Tensor = onnx::Unsqueeze[axes=[0]](%1264)
%1268 : Tensor = onnx::Unsqueeze[axes=[0]](%1246)
%1269 : Tensor = onnx::Concat[axis=0](%2488, %1267, %1268)
%1270 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%1251, %1269) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4079:0
%1271 : Long() = onnx::Constant[value={2}]()
%1272 : Long() = onnx::Mul(%1239, %1271)
%1275 : Tensor = onnx::Unsqueeze[axes=[0]](%1272)
%1276 : Tensor = onnx::Unsqueeze[axes=[0]](%1246)
%1277 : Tensor = onnx::Concat[axis=0](%2489, %1275, %1276)
%1278 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%1252, %1277) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%1279 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1278) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%1280 : Float(2:256, 256:1, 58:512) = onnx::Transpose[perm=[1, 2, 0]](%1270) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%1281 : Float(2:3364, 58:58, 58:1) = onnx::MatMul(%1262, %1280) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%1282 : Float(2:3364, 58:58, 58:1) = onnx::Softmax[axis=2](%1281) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1283 : Float(2:14848, 58:256, 256:1) = onnx::MatMul(%1282, %1279) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4130:0
%1284 : Float(58:512, 2:256, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1283) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%1285 : Tensor = onnx::Unsqueeze[axes=[0]](%1236)
%1286 : Tensor = onnx::Unsqueeze[axes=[0]](%1239)
%1287 : Tensor = onnx::Unsqueeze[axes=[0]](%1242)
%1288 : Tensor = onnx::Concat[axis=0](%1285, %1286, %1287)
%1289 : Float(58:512, 1:512, 512:1) = onnx::Reshape(%1284, %1288) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%1291 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1289, %2490) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1292 : Float(58:512, 1:512, 512:1) = onnx::Add(%1291, %decoder.layers.3.self_attn.out_proj.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1293 : Float(58:512, 1:512, 512:1) = onnx::Add(%1233, %1292) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:295:0
%1295 : Tensor = onnx::ReduceMean[axes=[-1]](%1293)
%1296 : FloatTensor = onnx::Sub(%1293, %1295)
%1297 : Tensor = onnx::Cast[to=1](%1296)
%1299 : Tensor = onnx::Pow(%1297, %2491)
%1300 : Tensor = onnx::ReduceMean[axes=[-1]](%1299)
%1301 : Float() = onnx::Constant[value={1e-05}]()
%1302 : FloatTensor = onnx::Add(%1300, %1301)
%1303 : Tensor = onnx::Sqrt(%1302)
%1304 : FloatTensor = onnx::Div(%1296, %1303)
%1305 : FloatTensor = onnx::Mul(%1304, %decoder.layers.3.norm1.weight)
%1306 : Float(58:512, 1:512, 512:1) = onnx::Add(%1305, %decoder.layers.3.norm1.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%1308 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1306, %2492) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1309 : Float(58:512, 1:512, 512:1) = onnx::Add(%1308, %decoder.layers.3.linear1.bias)
%1310 : Float(58:512, 1:512, 512:1) = onnx::Relu(%1309) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1312 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1310, %2493) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1313 : Float(58:512, 1:512, 512:1) = onnx::Add(%1312, %decoder.layers.3.linear2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1314 : Float(58:512, 1:512, 512:1) = onnx::Add(%1306, %1313) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:298:0
%1316 : Tensor = onnx::ReduceMean[axes=[-1]](%1314)
%1317 : FloatTensor = onnx::Sub(%1314, %1316)
%1318 : Tensor = onnx::Cast[to=1](%1317)
%1320 : Tensor = onnx::Pow(%1318, %2494)
%1321 : Tensor = onnx::ReduceMean[axes=[-1]](%1320)
%1322 : Float() = onnx::Constant[value={1e-05}]()
%1323 : FloatTensor = onnx::Add(%1321, %1322)
%1324 : Tensor = onnx::Sqrt(%1323)
%1325 : FloatTensor = onnx::Div(%1317, %1324)
%1326 : FloatTensor = onnx::Mul(%1325, %decoder.layers.3.norm2.weight)
%1327 : Float(58:512, 1:512, 512:1) = onnx::Add(%1326, %decoder.layers.3.norm2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%1328 : Tensor = onnx::Shape(%1327)
%1329 : Tensor = onnx::Constant[value={0}]()
%1330 : Long() = onnx::Gather[axis=0](%1328, %1329) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1331 : Tensor = onnx::Shape(%1327)
%1332 : Tensor = onnx::Constant[value={1}]()
%1333 : Long() = onnx::Gather[axis=0](%1331, %1332) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1334 : Tensor = onnx::Shape(%1327)
%1335 : Tensor = onnx::Constant[value={2}]()
%1336 : Long() = onnx::Gather[axis=0](%1334, %1335) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1337 : Long() = onnx::Constant[value={2}]()
%1338 : LongTensor = onnx::Div(%1336, %1337)
%1339 : Tensor = onnx::Cast[to=7](%1338)
%1340 : Long() = onnx::Cast[to=7](%1339) # /opt/conda/lib/python3.7/site-packages/torch/tensor.py:424:0
%1342 : Float(58:1536, 1:1536, 1536:1) = onnx::MatMul(%1327, %2495) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1343 : Float(58:1536, 1:1536, 1536:1) = onnx::Add(%1342, %decoder.layers.4.self_attn.in_proj_bias)
%1344 : Float(58:1536, 1:1536, 512:1), %1345 : Float(58:512, 1:512, 512:1), %1346 : Float(58:512, 1:512, 512:1) = onnx::Split[axis=-1, split=[512, 512, 512]](%1343) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%1347 : Float() = onnx::Constant[value={0.0625}]()
%1348 : Float(58:512, 1:512, 512:1) = onnx::Mul(%1344, %1347) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%1349 : Long() = onnx::Constant[value={2}]()
%1350 : Long() = onnx::Mul(%1333, %1349)
%1351 : Tensor = onnx::Unsqueeze[axes=[0]](%1330)
%1352 : Tensor = onnx::Unsqueeze[axes=[0]](%1350)
%1353 : Tensor = onnx::Unsqueeze[axes=[0]](%1340)
%1354 : Tensor = onnx::Concat[axis=0](%1351, %1352, %1353)
%1355 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%1348, %1354) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%1356 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1355) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%1357 : Long() = onnx::Constant[value={2}]()
%1358 : Long() = onnx::Mul(%1333, %1357)
%1361 : Tensor = onnx::Unsqueeze[axes=[0]](%1358)
%1362 : Tensor = onnx::Unsqueeze[axes=[0]](%1340)
%1363 : Tensor = onnx::Concat[axis=0](%2496, %1361, %1362)
%1364 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%1345, %1363) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4079:0
%1365 : Long() = onnx::Constant[value={2}]()
%1366 : Long() = onnx::Mul(%1333, %1365)
%1369 : Tensor = onnx::Unsqueeze[axes=[0]](%1366)
%1370 : Tensor = onnx::Unsqueeze[axes=[0]](%1340)
%1371 : Tensor = onnx::Concat[axis=0](%2497, %1369, %1370)
%1372 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%1346, %1371) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%1373 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1372) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%1374 : Float(2:256, 256:1, 58:512) = onnx::Transpose[perm=[1, 2, 0]](%1364) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%1375 : Float(2:3364, 58:58, 58:1) = onnx::MatMul(%1356, %1374) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%1376 : Float(2:3364, 58:58, 58:1) = onnx::Softmax[axis=2](%1375) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1377 : Float(2:14848, 58:256, 256:1) = onnx::MatMul(%1376, %1373) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4130:0
%1378 : Float(58:512, 2:256, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1377) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%1379 : Tensor = onnx::Unsqueeze[axes=[0]](%1330)
%1380 : Tensor = onnx::Unsqueeze[axes=[0]](%1333)
%1381 : Tensor = onnx::Unsqueeze[axes=[0]](%1336)
%1382 : Tensor = onnx::Concat[axis=0](%1379, %1380, %1381)
%1383 : Float(58:512, 1:512, 512:1) = onnx::Reshape(%1378, %1382) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%1385 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1383, %2498) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1386 : Float(58:512, 1:512, 512:1) = onnx::Add(%1385, %decoder.layers.4.self_attn.out_proj.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1387 : Float(58:512, 1:512, 512:1) = onnx::Add(%1327, %1386) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:295:0
%1389 : Tensor = onnx::ReduceMean[axes=[-1]](%1387)
%1390 : FloatTensor = onnx::Sub(%1387, %1389)
%1391 : Tensor = onnx::Cast[to=1](%1390)
%1393 : Tensor = onnx::Pow(%1391, %2499)
%1394 : Tensor = onnx::ReduceMean[axes=[-1]](%1393)
%1395 : Float() = onnx::Constant[value={1e-05}]()
%1396 : FloatTensor = onnx::Add(%1394, %1395)
%1397 : Tensor = onnx::Sqrt(%1396)
%1398 : FloatTensor = onnx::Div(%1390, %1397)
%1399 : FloatTensor = onnx::Mul(%1398, %decoder.layers.4.norm1.weight)
%1400 : Float(58:512, 1:512, 512:1) = onnx::Add(%1399, %decoder.layers.4.norm1.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%1402 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1400, %2500) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1403 : Float(58:512, 1:512, 512:1) = onnx::Add(%1402, %decoder.layers.4.linear1.bias)
%1404 : Float(58:512, 1:512, 512:1) = onnx::Relu(%1403) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1406 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1404, %2501) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1407 : Float(58:512, 1:512, 512:1) = onnx::Add(%1406, %decoder.layers.4.linear2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1408 : Float(58:512, 1:512, 512:1) = onnx::Add(%1400, %1407) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:298:0
%1410 : Tensor = onnx::ReduceMean[axes=[-1]](%1408)
%1411 : FloatTensor = onnx::Sub(%1408, %1410)
%1412 : Tensor = onnx::Cast[to=1](%1411)
%1414 : Tensor = onnx::Pow(%1412, %2502)
%1415 : Tensor = onnx::ReduceMean[axes=[-1]](%1414)
%1416 : Float() = onnx::Constant[value={1e-05}]()
%1417 : FloatTensor = onnx::Add(%1415, %1416)
%1418 : Tensor = onnx::Sqrt(%1417)
%1419 : FloatTensor = onnx::Div(%1411, %1418)
%1420 : FloatTensor = onnx::Mul(%1419, %decoder.layers.4.norm2.weight)
%1421 : Float(58:512, 1:512, 512:1) = onnx::Add(%1420, %decoder.layers.4.norm2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%1422 : Tensor = onnx::Shape(%1421)
%1423 : Tensor = onnx::Constant[value={0}]()
%1424 : Long() = onnx::Gather[axis=0](%1422, %1423) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1425 : Tensor = onnx::Shape(%1421)
%1426 : Tensor = onnx::Constant[value={1}]()
%1427 : Long() = onnx::Gather[axis=0](%1425, %1426) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1428 : Tensor = onnx::Shape(%1421)
%1429 : Tensor = onnx::Constant[value={2}]()
%1430 : Long() = onnx::Gather[axis=0](%1428, %1429) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1431 : Long() = onnx::Constant[value={2}]()
%1432 : LongTensor = onnx::Div(%1430, %1431)
%1433 : Tensor = onnx::Cast[to=7](%1432)
%1434 : Long() = onnx::Cast[to=7](%1433) # /opt/conda/lib/python3.7/site-packages/torch/tensor.py:424:0
%1436 : Float(58:1536, 1:1536, 1536:1) = onnx::MatMul(%1421, %2503) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1437 : Float(58:1536, 1:1536, 1536:1) = onnx::Add(%1436, %decoder.layers.5.self_attn.in_proj_bias)
%1438 : Float(58:1536, 1:1536, 512:1), %1439 : Float(58:512, 1:512, 512:1), %1440 : Float(58:512, 1:512, 512:1) = onnx::Split[axis=-1, split=[512, 512, 512]](%1437) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%1441 : Float() = onnx::Constant[value={0.0625}]()
%1442 : Float(58:512, 1:512, 512:1) = onnx::Mul(%1438, %1441) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%1443 : Long() = onnx::Constant[value={2}]()
%1444 : Long() = onnx::Mul(%1427, %1443)
%1445 : Tensor = onnx::Unsqueeze[axes=[0]](%1424)
%1446 : Tensor = onnx::Unsqueeze[axes=[0]](%1444)
%1447 : Tensor = onnx::Unsqueeze[axes=[0]](%1434)
%1448 : Tensor = onnx::Concat[axis=0](%1445, %1446, %1447)
%1449 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%1442, %1448) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%1450 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1449) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%1451 : Long() = onnx::Constant[value={2}]()
%1452 : Long() = onnx::Mul(%1427, %1451)
%1455 : Tensor = onnx::Unsqueeze[axes=[0]](%1452)
%1456 : Tensor = onnx::Unsqueeze[axes=[0]](%1434)
%1457 : Tensor = onnx::Concat[axis=0](%2504, %1455, %1456)
%1458 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%1439, %1457) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4079:0
%1459 : Long() = onnx::Constant[value={2}]()
%1460 : Long() = onnx::Mul(%1427, %1459)
%1463 : Tensor = onnx::Unsqueeze[axes=[0]](%1460)
%1464 : Tensor = onnx::Unsqueeze[axes=[0]](%1434)
%1465 : Tensor = onnx::Concat[axis=0](%2505, %1463, %1464)
%1466 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%1440, %1465) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%1467 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1466) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%1468 : Float(2:256, 256:1, 58:512) = onnx::Transpose[perm=[1, 2, 0]](%1458) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%1469 : Float(2:3364, 58:58, 58:1) = onnx::MatMul(%1450, %1468) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%1470 : Float(2:3364, 58:58, 58:1) = onnx::Softmax[axis=2](%1469) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1471 : Float(2:14848, 58:256, 256:1) = onnx::MatMul(%1470, %1467) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4130:0
%1472 : Float(58:512, 2:256, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1471) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%1473 : Tensor = onnx::Unsqueeze[axes=[0]](%1424)
%1474 : Tensor = onnx::Unsqueeze[axes=[0]](%1427)
%1475 : Tensor = onnx::Unsqueeze[axes=[0]](%1430)
%1476 : Tensor = onnx::Concat[axis=0](%1473, %1474, %1475)
%1477 : Float(58:512, 1:512, 512:1) = onnx::Reshape(%1472, %1476) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%1479 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1477, %2506) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1480 : Float(58:512, 1:512, 512:1) = onnx::Add(%1479, %decoder.layers.5.self_attn.out_proj.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1481 : Float(58:512, 1:512, 512:1) = onnx::Add(%1421, %1480) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:295:0
%1483 : Tensor = onnx::ReduceMean[axes=[-1]](%1481)
%1484 : FloatTensor = onnx::Sub(%1481, %1483)
%1485 : Tensor = onnx::Cast[to=1](%1484)
%1487 : Tensor = onnx::Pow(%1485, %2507)
%1488 : Tensor = onnx::ReduceMean[axes=[-1]](%1487)
%1489 : Float() = onnx::Constant[value={1e-05}]()
%1490 : FloatTensor = onnx::Add(%1488, %1489)
%1491 : Tensor = onnx::Sqrt(%1490)
%1492 : FloatTensor = onnx::Div(%1484, %1491)
%1493 : FloatTensor = onnx::Mul(%1492, %decoder.layers.5.norm1.weight)
%1494 : Float(58:512, 1:512, 512:1) = onnx::Add(%1493, %decoder.layers.5.norm1.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%1496 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1494, %2508) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1497 : Float(58:512, 1:512, 512:1) = onnx::Add(%1496, %decoder.layers.5.linear1.bias)
%1498 : Float(58:512, 1:512, 512:1) = onnx::Relu(%1497) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1500 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1498, %2509) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1501 : Float(58:512, 1:512, 512:1) = onnx::Add(%1500, %decoder.layers.5.linear2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1502 : Float(58:512, 1:512, 512:1) = onnx::Add(%1494, %1501) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:298:0
%1504 : Tensor = onnx::ReduceMean[axes=[-1]](%1502)
%1505 : FloatTensor = onnx::Sub(%1502, %1504)
%1506 : Tensor = onnx::Cast[to=1](%1505)
%1508 : Tensor = onnx::Pow(%1506, %2510)
%1509 : Tensor = onnx::ReduceMean[axes=[-1]](%1508)
%1510 : Float() = onnx::Constant[value={1e-05}]()
%1511 : FloatTensor = onnx::Add(%1509, %1510)
%1512 : Tensor = onnx::Sqrt(%1511)
%1513 : FloatTensor = onnx::Div(%1505, %1512)
%1514 : FloatTensor = onnx::Mul(%1513, %decoder.layers.5.norm2.weight)
%1515 : Float(58:512, 1:512, 512:1) = onnx::Add(%1514, %decoder.layers.5.norm2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%1516 : Tensor = onnx::Shape(%1515)
%1517 : Tensor = onnx::Constant[value={0}]()
%1518 : Long() = onnx::Gather[axis=0](%1516, %1517) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1519 : Tensor = onnx::Shape(%1515)
%1520 : Tensor = onnx::Constant[value={1}]()
%1521 : Long() = onnx::Gather[axis=0](%1519, %1520) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1522 : Tensor = onnx::Shape(%1515)
%1523 : Tensor = onnx::Constant[value={2}]()
%1524 : Long() = onnx::Gather[axis=0](%1522, %1523) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1525 : Long() = onnx::Constant[value={2}]()
%1526 : LongTensor = onnx::Div(%1524, %1525)
%1527 : Tensor = onnx::Cast[to=7](%1526)
%1528 : Long() = onnx::Cast[to=7](%1527) # /opt/conda/lib/python3.7/site-packages/torch/tensor.py:424:0
%1530 : Float(58:1536, 1:1536, 1536:1) = onnx::MatMul(%1515, %2511) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1531 : Float(58:1536, 1:1536, 1536:1) = onnx::Add(%1530, %decoder.layers.6.self_attn.in_proj_bias)
%1532 : Float(58:1536, 1:1536, 512:1), %1533 : Float(58:512, 1:512, 512:1), %1534 : Float(58:512, 1:512, 512:1) = onnx::Split[axis=-1, split=[512, 512, 512]](%1531) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%1535 : Float() = onnx::Constant[value={0.0625}]()
%1536 : Float(58:512, 1:512, 512:1) = onnx::Mul(%1532, %1535) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%1537 : Long() = onnx::Constant[value={2}]()
%1538 : Long() = onnx::Mul(%1521, %1537)
%1539 : Tensor = onnx::Unsqueeze[axes=[0]](%1518)
%1540 : Tensor = onnx::Unsqueeze[axes=[0]](%1538)
%1541 : Tensor = onnx::Unsqueeze[axes=[0]](%1528)
%1542 : Tensor = onnx::Concat[axis=0](%1539, %1540, %1541)
%1543 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%1536, %1542) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%1544 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1543) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%1545 : Long() = onnx::Constant[value={2}]()
%1546 : Long() = onnx::Mul(%1521, %1545)
%1549 : Tensor = onnx::Unsqueeze[axes=[0]](%1546)
%1550 : Tensor = onnx::Unsqueeze[axes=[0]](%1528)
%1551 : Tensor = onnx::Concat[axis=0](%2512, %1549, %1550)
%1552 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%1533, %1551) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4079:0
%1553 : Long() = onnx::Constant[value={2}]()
%1554 : Long() = onnx::Mul(%1521, %1553)
%1557 : Tensor = onnx::Unsqueeze[axes=[0]](%1554)
%1558 : Tensor = onnx::Unsqueeze[axes=[0]](%1528)
%1559 : Tensor = onnx::Concat[axis=0](%2513, %1557, %1558)
%1560 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%1534, %1559) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%1561 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1560) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%1562 : Float(2:256, 256:1, 58:512) = onnx::Transpose[perm=[1, 2, 0]](%1552) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%1563 : Float(2:3364, 58:58, 58:1) = onnx::MatMul(%1544, %1562) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%1564 : Float(2:3364, 58:58, 58:1) = onnx::Softmax[axis=2](%1563) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1565 : Float(2:14848, 58:256, 256:1) = onnx::MatMul(%1564, %1561) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4130:0
%1566 : Float(58:512, 2:256, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1565) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%1567 : Tensor = onnx::Unsqueeze[axes=[0]](%1518)
%1568 : Tensor = onnx::Unsqueeze[axes=[0]](%1521)
%1569 : Tensor = onnx::Unsqueeze[axes=[0]](%1524)
%1570 : Tensor = onnx::Concat[axis=0](%1567, %1568, %1569)
%1571 : Float(58:512, 1:512, 512:1) = onnx::Reshape(%1566, %1570) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%1573 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1571, %2514) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1574 : Float(58:512, 1:512, 512:1) = onnx::Add(%1573, %decoder.layers.6.self_attn.out_proj.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1575 : Float(58:512, 1:512, 512:1) = onnx::Add(%1515, %1574) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:295:0
%1577 : Tensor = onnx::ReduceMean[axes=[-1]](%1575)
%1578 : FloatTensor = onnx::Sub(%1575, %1577)
%1579 : Tensor = onnx::Cast[to=1](%1578)
%1581 : Tensor = onnx::Pow(%1579, %2515)
%1582 : Tensor = onnx::ReduceMean[axes=[-1]](%1581)
%1583 : Float() = onnx::Constant[value={1e-05}]()
%1584 : FloatTensor = onnx::Add(%1582, %1583)
%1585 : Tensor = onnx::Sqrt(%1584)
%1586 : FloatTensor = onnx::Div(%1578, %1585)
%1587 : FloatTensor = onnx::Mul(%1586, %decoder.layers.6.norm1.weight)
%1588 : Float(58:512, 1:512, 512:1) = onnx::Add(%1587, %decoder.layers.6.norm1.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%1590 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1588, %2516) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1591 : Float(58:512, 1:512, 512:1) = onnx::Add(%1590, %decoder.layers.6.linear1.bias)
%1592 : Float(58:512, 1:512, 512:1) = onnx::Relu(%1591) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1594 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1592, %2517) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1595 : Float(58:512, 1:512, 512:1) = onnx::Add(%1594, %decoder.layers.6.linear2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1596 : Float(58:512, 1:512, 512:1) = onnx::Add(%1588, %1595) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:298:0
%1598 : Tensor = onnx::ReduceMean[axes=[-1]](%1596)
%1599 : FloatTensor = onnx::Sub(%1596, %1598)
%1600 : Tensor = onnx::Cast[to=1](%1599)
%1602 : Tensor = onnx::Pow(%1600, %2518)
%1603 : Tensor = onnx::ReduceMean[axes=[-1]](%1602)
%1604 : Float() = onnx::Constant[value={1e-05}]()
%1605 : FloatTensor = onnx::Add(%1603, %1604)
%1606 : Tensor = onnx::Sqrt(%1605)
%1607 : FloatTensor = onnx::Div(%1599, %1606)
%1608 : FloatTensor = onnx::Mul(%1607, %decoder.layers.6.norm2.weight)
%1609 : Float(58:512, 1:512, 512:1) = onnx::Add(%1608, %decoder.layers.6.norm2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%1610 : Tensor = onnx::Shape(%1609)
%1611 : Tensor = onnx::Constant[value={0}]()
%1612 : Long() = onnx::Gather[axis=0](%1610, %1611) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1613 : Tensor = onnx::Shape(%1609)
%1614 : Tensor = onnx::Constant[value={1}]()
%1615 : Long() = onnx::Gather[axis=0](%1613, %1614) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1616 : Tensor = onnx::Shape(%1609)
%1617 : Tensor = onnx::Constant[value={2}]()
%1618 : Long() = onnx::Gather[axis=0](%1616, %1617) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1619 : Long() = onnx::Constant[value={2}]()
%1620 : LongTensor = onnx::Div(%1618, %1619)
%1621 : Tensor = onnx::Cast[to=7](%1620)
%1622 : Long() = onnx::Cast[to=7](%1621) # /opt/conda/lib/python3.7/site-packages/torch/tensor.py:424:0
%1624 : Float(58:1536, 1:1536, 1536:1) = onnx::MatMul(%1609, %2519) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1625 : Float(58:1536, 1:1536, 1536:1) = onnx::Add(%1624, %decoder.layers.7.self_attn.in_proj_bias)
%1626 : Float(58:1536, 1:1536, 512:1), %1627 : Float(58:512, 1:512, 512:1), %1628 : Float(58:512, 1:512, 512:1) = onnx::Split[axis=-1, split=[512, 512, 512]](%1625) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%1629 : Float() = onnx::Constant[value={0.0625}]()
%1630 : Float(58:512, 1:512, 512:1) = onnx::Mul(%1626, %1629) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%1631 : Long() = onnx::Constant[value={2}]()
%1632 : Long() = onnx::Mul(%1615, %1631)
%1633 : Tensor = onnx::Unsqueeze[axes=[0]](%1612)
%1634 : Tensor = onnx::Unsqueeze[axes=[0]](%1632)
%1635 : Tensor = onnx::Unsqueeze[axes=[0]](%1622)
%1636 : Tensor = onnx::Concat[axis=0](%1633, %1634, %1635)
%1637 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%1630, %1636) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%1638 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1637) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%1639 : Long() = onnx::Constant[value={2}]()
%1640 : Long() = onnx::Mul(%1615, %1639)
%1643 : Tensor = onnx::Unsqueeze[axes=[0]](%1640)
%1644 : Tensor = onnx::Unsqueeze[axes=[0]](%1622)
%1645 : Tensor = onnx::Concat[axis=0](%2520, %1643, %1644)
%1646 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%1627, %1645) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4079:0
%1647 : Long() = onnx::Constant[value={2}]()
%1648 : Long() = onnx::Mul(%1615, %1647)
%1651 : Tensor = onnx::Unsqueeze[axes=[0]](%1648)
%1652 : Tensor = onnx::Unsqueeze[axes=[0]](%1622)
%1653 : Tensor = onnx::Concat[axis=0](%2521, %1651, %1652)
%1654 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%1628, %1653) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%1655 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1654) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%1656 : Float(2:256, 256:1, 58:512) = onnx::Transpose[perm=[1, 2, 0]](%1646) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%1657 : Float(2:3364, 58:58, 58:1) = onnx::MatMul(%1638, %1656) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%1658 : Float(2:3364, 58:58, 58:1) = onnx::Softmax[axis=2](%1657) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1659 : Float(2:14848, 58:256, 256:1) = onnx::MatMul(%1658, %1655) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4130:0
%1660 : Float(58:512, 2:256, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1659) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%1661 : Tensor = onnx::Unsqueeze[axes=[0]](%1612)
%1662 : Tensor = onnx::Unsqueeze[axes=[0]](%1615)
%1663 : Tensor = onnx::Unsqueeze[axes=[0]](%1618)
%1664 : Tensor = onnx::Concat[axis=0](%1661, %1662, %1663)
%1665 : Float(58:512, 1:512, 512:1) = onnx::Reshape(%1660, %1664) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%1667 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1665, %2522) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1668 : Float(58:512, 1:512, 512:1) = onnx::Add(%1667, %decoder.layers.7.self_attn.out_proj.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1669 : Float(58:512, 1:512, 512:1) = onnx::Add(%1609, %1668) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:295:0
%1671 : Tensor = onnx::ReduceMean[axes=[-1]](%1669)
%1672 : FloatTensor = onnx::Sub(%1669, %1671)
%1673 : Tensor = onnx::Cast[to=1](%1672)
%1675 : Tensor = onnx::Pow(%1673, %2523)
%1676 : Tensor = onnx::ReduceMean[axes=[-1]](%1675)
%1677 : Float() = onnx::Constant[value={1e-05}]()
%1678 : FloatTensor = onnx::Add(%1676, %1677)
%1679 : Tensor = onnx::Sqrt(%1678)
%1680 : FloatTensor = onnx::Div(%1672, %1679)
%1681 : FloatTensor = onnx::Mul(%1680, %decoder.layers.7.norm1.weight)
%1682 : Float(58:512, 1:512, 512:1) = onnx::Add(%1681, %decoder.layers.7.norm1.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%1684 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1682, %2524) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1685 : Float(58:512, 1:512, 512:1) = onnx::Add(%1684, %decoder.layers.7.linear1.bias)
%1686 : Float(58:512, 1:512, 512:1) = onnx::Relu(%1685) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1688 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1686, %2525) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1689 : Float(58:512, 1:512, 512:1) = onnx::Add(%1688, %decoder.layers.7.linear2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1690 : Float(58:512, 1:512, 512:1) = onnx::Add(%1682, %1689) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:298:0
%1692 : Tensor = onnx::ReduceMean[axes=[-1]](%1690)
%1693 : FloatTensor = onnx::Sub(%1690, %1692)
%1694 : Tensor = onnx::Cast[to=1](%1693)
%1696 : Tensor = onnx::Pow(%1694, %2526)
%1697 : Tensor = onnx::ReduceMean[axes=[-1]](%1696)
%1698 : Float() = onnx::Constant[value={1e-05}]()
%1699 : FloatTensor = onnx::Add(%1697, %1698)
%1700 : Tensor = onnx::Sqrt(%1699)
%1701 : FloatTensor = onnx::Div(%1693, %1700)
%1702 : FloatTensor = onnx::Mul(%1701, %decoder.layers.7.norm2.weight)
%1703 : Float(58:512, 1:512, 512:1) = onnx::Add(%1702, %decoder.layers.7.norm2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%1704 : Tensor = onnx::Shape(%1703)
%1705 : Tensor = onnx::Constant[value={0}]()
%1706 : Long() = onnx::Gather[axis=0](%1704, %1705) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1707 : Tensor = onnx::Shape(%1703)
%1708 : Tensor = onnx::Constant[value={1}]()
%1709 : Long() = onnx::Gather[axis=0](%1707, %1708) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1710 : Tensor = onnx::Shape(%1703)
%1711 : Tensor = onnx::Constant[value={2}]()
%1712 : Long() = onnx::Gather[axis=0](%1710, %1711) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1713 : Long() = onnx::Constant[value={2}]()
%1714 : LongTensor = onnx::Div(%1712, %1713)
%1715 : Tensor = onnx::Cast[to=7](%1714)
%1716 : Long() = onnx::Cast[to=7](%1715) # /opt/conda/lib/python3.7/site-packages/torch/tensor.py:424:0
%1718 : Float(58:1536, 1:1536, 1536:1) = onnx::MatMul(%1703, %2527) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1719 : Float(58:1536, 1:1536, 1536:1) = onnx::Add(%1718, %decoder.layers.8.self_attn.in_proj_bias)
%1720 : Float(58:1536, 1:1536, 512:1), %1721 : Float(58:512, 1:512, 512:1), %1722 : Float(58:512, 1:512, 512:1) = onnx::Split[axis=-1, split=[512, 512, 512]](%1719) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%1723 : Float() = onnx::Constant[value={0.0625}]()
%1724 : Float(58:512, 1:512, 512:1) = onnx::Mul(%1720, %1723) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%1725 : Long() = onnx::Constant[value={2}]()
%1726 : Long() = onnx::Mul(%1709, %1725)
%1727 : Tensor = onnx::Unsqueeze[axes=[0]](%1706)
%1728 : Tensor = onnx::Unsqueeze[axes=[0]](%1726)
%1729 : Tensor = onnx::Unsqueeze[axes=[0]](%1716)
%1730 : Tensor = onnx::Concat[axis=0](%1727, %1728, %1729)
%1731 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%1724, %1730) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%1732 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1731) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%1733 : Long() = onnx::Constant[value={2}]()
%1734 : Long() = onnx::Mul(%1709, %1733)
%1737 : Tensor = onnx::Unsqueeze[axes=[0]](%1734)
%1738 : Tensor = onnx::Unsqueeze[axes=[0]](%1716)
%1739 : Tensor = onnx::Concat[axis=0](%2528, %1737, %1738)
%1740 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%1721, %1739) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4079:0
%1741 : Long() = onnx::Constant[value={2}]()
%1742 : Long() = onnx::Mul(%1709, %1741)
%1745 : Tensor = onnx::Unsqueeze[axes=[0]](%1742)
%1746 : Tensor = onnx::Unsqueeze[axes=[0]](%1716)
%1747 : Tensor = onnx::Concat[axis=0](%2529, %1745, %1746)
%1748 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%1722, %1747) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%1749 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1748) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%1750 : Float(2:256, 256:1, 58:512) = onnx::Transpose[perm=[1, 2, 0]](%1740) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%1751 : Float(2:3364, 58:58, 58:1) = onnx::MatMul(%1732, %1750) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%1752 : Float(2:3364, 58:58, 58:1) = onnx::Softmax[axis=2](%1751) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1753 : Float(2:14848, 58:256, 256:1) = onnx::MatMul(%1752, %1749) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4130:0
%1754 : Float(58:512, 2:256, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1753) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%1755 : Tensor = onnx::Unsqueeze[axes=[0]](%1706)
%1756 : Tensor = onnx::Unsqueeze[axes=[0]](%1709)
%1757 : Tensor = onnx::Unsqueeze[axes=[0]](%1712)
%1758 : Tensor = onnx::Concat[axis=0](%1755, %1756, %1757)
%1759 : Float(58:512, 1:512, 512:1) = onnx::Reshape(%1754, %1758) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%1761 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1759, %2530) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1762 : Float(58:512, 1:512, 512:1) = onnx::Add(%1761, %decoder.layers.8.self_attn.out_proj.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1763 : Float(58:512, 1:512, 512:1) = onnx::Add(%1703, %1762) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:295:0
%1765 : Tensor = onnx::ReduceMean[axes=[-1]](%1763)
%1766 : FloatTensor = onnx::Sub(%1763, %1765)
%1767 : Tensor = onnx::Cast[to=1](%1766)
%1769 : Tensor = onnx::Pow(%1767, %2531)
%1770 : Tensor = onnx::ReduceMean[axes=[-1]](%1769)
%1771 : Float() = onnx::Constant[value={1e-05}]()
%1772 : FloatTensor = onnx::Add(%1770, %1771)
%1773 : Tensor = onnx::Sqrt(%1772)
%1774 : FloatTensor = onnx::Div(%1766, %1773)
%1775 : FloatTensor = onnx::Mul(%1774, %decoder.layers.8.norm1.weight)
%1776 : Float(58:512, 1:512, 512:1) = onnx::Add(%1775, %decoder.layers.8.norm1.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%1778 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1776, %2532) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1779 : Float(58:512, 1:512, 512:1) = onnx::Add(%1778, %decoder.layers.8.linear1.bias)
%1780 : Float(58:512, 1:512, 512:1) = onnx::Relu(%1779) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1782 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1780, %2533) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1783 : Float(58:512, 1:512, 512:1) = onnx::Add(%1782, %decoder.layers.8.linear2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1784 : Float(58:512, 1:512, 512:1) = onnx::Add(%1776, %1783) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:298:0
%1786 : Tensor = onnx::ReduceMean[axes=[-1]](%1784)
%1787 : FloatTensor = onnx::Sub(%1784, %1786)
%1788 : Tensor = onnx::Cast[to=1](%1787)
%1790 : Tensor = onnx::Pow(%1788, %2534)
%1791 : Tensor = onnx::ReduceMean[axes=[-1]](%1790)
%1792 : Float() = onnx::Constant[value={1e-05}]()
%1793 : FloatTensor = onnx::Add(%1791, %1792)
%1794 : Tensor = onnx::Sqrt(%1793)
%1795 : FloatTensor = onnx::Div(%1787, %1794)
%1796 : FloatTensor = onnx::Mul(%1795, %decoder.layers.8.norm2.weight)
%1797 : Float(58:512, 1:512, 512:1) = onnx::Add(%1796, %decoder.layers.8.norm2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%1798 : Tensor = onnx::Shape(%1797)
%1799 : Tensor = onnx::Constant[value={0}]()
%1800 : Long() = onnx::Gather[axis=0](%1798, %1799) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1801 : Tensor = onnx::Shape(%1797)
%1802 : Tensor = onnx::Constant[value={1}]()
%1803 : Long() = onnx::Gather[axis=0](%1801, %1802) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1804 : Tensor = onnx::Shape(%1797)
%1805 : Tensor = onnx::Constant[value={2}]()
%1806 : Long() = onnx::Gather[axis=0](%1804, %1805) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1807 : Long() = onnx::Constant[value={2}]()
%1808 : LongTensor = onnx::Div(%1806, %1807)
%1809 : Tensor = onnx::Cast[to=7](%1808)
%1810 : Long() = onnx::Cast[to=7](%1809) # /opt/conda/lib/python3.7/site-packages/torch/tensor.py:424:0
%1812 : Float(58:1536, 1:1536, 1536:1) = onnx::MatMul(%1797, %2535) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1813 : Float(58:1536, 1:1536, 1536:1) = onnx::Add(%1812, %decoder.layers.9.self_attn.in_proj_bias)
%1814 : Float(58:1536, 1:1536, 512:1), %1815 : Float(58:512, 1:512, 512:1), %1816 : Float(58:512, 1:512, 512:1) = onnx::Split[axis=-1, split=[512, 512, 512]](%1813) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%1817 : Float() = onnx::Constant[value={0.0625}]()
%1818 : Float(58:512, 1:512, 512:1) = onnx::Mul(%1814, %1817) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%1819 : Long() = onnx::Constant[value={2}]()
%1820 : Long() = onnx::Mul(%1803, %1819)
%1821 : Tensor = onnx::Unsqueeze[axes=[0]](%1800)
%1822 : Tensor = onnx::Unsqueeze[axes=[0]](%1820)
%1823 : Tensor = onnx::Unsqueeze[axes=[0]](%1810)
%1824 : Tensor = onnx::Concat[axis=0](%1821, %1822, %1823)
%1825 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%1818, %1824) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%1826 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1825) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%1827 : Long() = onnx::Constant[value={2}]()
%1828 : Long() = onnx::Mul(%1803, %1827)
%1831 : Tensor = onnx::Unsqueeze[axes=[0]](%1828)
%1832 : Tensor = onnx::Unsqueeze[axes=[0]](%1810)
%1833 : Tensor = onnx::Concat[axis=0](%2536, %1831, %1832)
%1834 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%1815, %1833) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4079:0
%1835 : Long() = onnx::Constant[value={2}]()
%1836 : Long() = onnx::Mul(%1803, %1835)
%1839 : Tensor = onnx::Unsqueeze[axes=[0]](%1836)
%1840 : Tensor = onnx::Unsqueeze[axes=[0]](%1810)
%1841 : Tensor = onnx::Concat[axis=0](%2537, %1839, %1840)
%1842 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%1816, %1841) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%1843 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1842) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%1844 : Float(2:256, 256:1, 58:512) = onnx::Transpose[perm=[1, 2, 0]](%1834) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%1845 : Float(2:3364, 58:58, 58:1) = onnx::MatMul(%1826, %1844) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%1846 : Float(2:3364, 58:58, 58:1) = onnx::Softmax[axis=2](%1845) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1847 : Float(2:14848, 58:256, 256:1) = onnx::MatMul(%1846, %1843) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4130:0
%1848 : Float(58:512, 2:256, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1847) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%1849 : Tensor = onnx::Unsqueeze[axes=[0]](%1800)
%1850 : Tensor = onnx::Unsqueeze[axes=[0]](%1803)
%1851 : Tensor = onnx::Unsqueeze[axes=[0]](%1806)
%1852 : Tensor = onnx::Concat[axis=0](%1849, %1850, %1851)
%1853 : Float(58:512, 1:512, 512:1) = onnx::Reshape(%1848, %1852) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%1855 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1853, %2538) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1856 : Float(58:512, 1:512, 512:1) = onnx::Add(%1855, %decoder.layers.9.self_attn.out_proj.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1857 : Float(58:512, 1:512, 512:1) = onnx::Add(%1797, %1856) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:295:0
%1859 : Tensor = onnx::ReduceMean[axes=[-1]](%1857)
%1860 : FloatTensor = onnx::Sub(%1857, %1859)
%1861 : Tensor = onnx::Cast[to=1](%1860)
%1863 : Tensor = onnx::Pow(%1861, %2539)
%1864 : Tensor = onnx::ReduceMean[axes=[-1]](%1863)
%1865 : Float() = onnx::Constant[value={1e-05}]()
%1866 : FloatTensor = onnx::Add(%1864, %1865)
%1867 : Tensor = onnx::Sqrt(%1866)
%1868 : FloatTensor = onnx::Div(%1860, %1867)
%1869 : FloatTensor = onnx::Mul(%1868, %decoder.layers.9.norm1.weight)
%1870 : Float(58:512, 1:512, 512:1) = onnx::Add(%1869, %decoder.layers.9.norm1.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%1872 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1870, %2540) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1873 : Float(58:512, 1:512, 512:1) = onnx::Add(%1872, %decoder.layers.9.linear1.bias)
%1874 : Float(58:512, 1:512, 512:1) = onnx::Relu(%1873) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1876 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1874, %2541) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1877 : Float(58:512, 1:512, 512:1) = onnx::Add(%1876, %decoder.layers.9.linear2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1878 : Float(58:512, 1:512, 512:1) = onnx::Add(%1870, %1877) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:298:0
%1880 : Tensor = onnx::ReduceMean[axes=[-1]](%1878)
%1881 : FloatTensor = onnx::Sub(%1878, %1880)
%1882 : Tensor = onnx::Cast[to=1](%1881)
%1884 : Tensor = onnx::Pow(%1882, %2542)
%1885 : Tensor = onnx::ReduceMean[axes=[-1]](%1884)
%1886 : Float() = onnx::Constant[value={1e-05}]()
%1887 : FloatTensor = onnx::Add(%1885, %1886)
%1888 : Tensor = onnx::Sqrt(%1887)
%1889 : FloatTensor = onnx::Div(%1881, %1888)
%1890 : FloatTensor = onnx::Mul(%1889, %decoder.layers.9.norm2.weight)
%1891 : Float(58:512, 1:512, 512:1) = onnx::Add(%1890, %decoder.layers.9.norm2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%1892 : Tensor = onnx::Shape(%1891)
%1893 : Tensor = onnx::Constant[value={0}]()
%1894 : Long() = onnx::Gather[axis=0](%1892, %1893) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1895 : Tensor = onnx::Shape(%1891)
%1896 : Tensor = onnx::Constant[value={1}]()
%1897 : Long() = onnx::Gather[axis=0](%1895, %1896) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1898 : Tensor = onnx::Shape(%1891)
%1899 : Tensor = onnx::Constant[value={2}]()
%1900 : Long() = onnx::Gather[axis=0](%1898, %1899) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1901 : Long() = onnx::Constant[value={2}]()
%1902 : LongTensor = onnx::Div(%1900, %1901)
%1903 : Tensor = onnx::Cast[to=7](%1902)
%1904 : Long() = onnx::Cast[to=7](%1903) # /opt/conda/lib/python3.7/site-packages/torch/tensor.py:424:0
%1906 : Float(58:1536, 1:1536, 1536:1) = onnx::MatMul(%1891, %2543) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1907 : Float(58:1536, 1:1536, 1536:1) = onnx::Add(%1906, %decoder.layers.10.self_attn.in_proj_bias)
%1908 : Float(58:1536, 1:1536, 512:1), %1909 : Float(58:512, 1:512, 512:1), %1910 : Float(58:512, 1:512, 512:1) = onnx::Split[axis=-1, split=[512, 512, 512]](%1907) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%1911 : Float() = onnx::Constant[value={0.0625}]()
%1912 : Float(58:512, 1:512, 512:1) = onnx::Mul(%1908, %1911) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%1913 : Long() = onnx::Constant[value={2}]()
%1914 : Long() = onnx::Mul(%1897, %1913)
%1915 : Tensor = onnx::Unsqueeze[axes=[0]](%1894)
%1916 : Tensor = onnx::Unsqueeze[axes=[0]](%1914)
%1917 : Tensor = onnx::Unsqueeze[axes=[0]](%1904)
%1918 : Tensor = onnx::Concat[axis=0](%1915, %1916, %1917)
%1919 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%1912, %1918) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%1920 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1919) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%1921 : Long() = onnx::Constant[value={2}]()
%1922 : Long() = onnx::Mul(%1897, %1921)
%1925 : Tensor = onnx::Unsqueeze[axes=[0]](%1922)
%1926 : Tensor = onnx::Unsqueeze[axes=[0]](%1904)
%1927 : Tensor = onnx::Concat[axis=0](%2544, %1925, %1926)
%1928 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%1909, %1927) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4079:0
%1929 : Long() = onnx::Constant[value={2}]()
%1930 : Long() = onnx::Mul(%1897, %1929)
%1933 : Tensor = onnx::Unsqueeze[axes=[0]](%1930)
%1934 : Tensor = onnx::Unsqueeze[axes=[0]](%1904)
%1935 : Tensor = onnx::Concat[axis=0](%2545, %1933, %1934)
%1936 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%1910, %1935) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%1937 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1936) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%1938 : Float(2:256, 256:1, 58:512) = onnx::Transpose[perm=[1, 2, 0]](%1928) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%1939 : Float(2:3364, 58:58, 58:1) = onnx::MatMul(%1920, %1938) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%1940 : Float(2:3364, 58:58, 58:1) = onnx::Softmax[axis=2](%1939) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1941 : Float(2:14848, 58:256, 256:1) = onnx::MatMul(%1940, %1937) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4130:0
%1942 : Float(58:512, 2:256, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%1941) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%1943 : Tensor = onnx::Unsqueeze[axes=[0]](%1894)
%1944 : Tensor = onnx::Unsqueeze[axes=[0]](%1897)
%1945 : Tensor = onnx::Unsqueeze[axes=[0]](%1900)
%1946 : Tensor = onnx::Concat[axis=0](%1943, %1944, %1945)
%1947 : Float(58:512, 1:512, 512:1) = onnx::Reshape(%1942, %1946) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%1949 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1947, %2546) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1950 : Float(58:512, 1:512, 512:1) = onnx::Add(%1949, %decoder.layers.10.self_attn.out_proj.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1951 : Float(58:512, 1:512, 512:1) = onnx::Add(%1891, %1950) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:295:0
%1953 : Tensor = onnx::ReduceMean[axes=[-1]](%1951)
%1954 : FloatTensor = onnx::Sub(%1951, %1953)
%1955 : Tensor = onnx::Cast[to=1](%1954)
%1957 : Tensor = onnx::Pow(%1955, %2547)
%1958 : Tensor = onnx::ReduceMean[axes=[-1]](%1957)
%1959 : Float() = onnx::Constant[value={1e-05}]()
%1960 : FloatTensor = onnx::Add(%1958, %1959)
%1961 : Tensor = onnx::Sqrt(%1960)
%1962 : FloatTensor = onnx::Div(%1954, %1961)
%1963 : FloatTensor = onnx::Mul(%1962, %decoder.layers.10.norm1.weight)
%1964 : Float(58:512, 1:512, 512:1) = onnx::Add(%1963, %decoder.layers.10.norm1.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%1966 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1964, %2548) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1967 : Float(58:512, 1:512, 512:1) = onnx::Add(%1966, %decoder.layers.10.linear1.bias)
%1968 : Float(58:512, 1:512, 512:1) = onnx::Relu(%1967) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1970 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%1968, %2549) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%1971 : Float(58:512, 1:512, 512:1) = onnx::Add(%1970, %decoder.layers.10.linear2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%1972 : Float(58:512, 1:512, 512:1) = onnx::Add(%1964, %1971) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:298:0
%1974 : Tensor = onnx::ReduceMean[axes=[-1]](%1972)
%1975 : FloatTensor = onnx::Sub(%1972, %1974)
%1976 : Tensor = onnx::Cast[to=1](%1975)
%1978 : Tensor = onnx::Pow(%1976, %2550)
%1979 : Tensor = onnx::ReduceMean[axes=[-1]](%1978)
%1980 : Float() = onnx::Constant[value={1e-05}]()
%1981 : FloatTensor = onnx::Add(%1979, %1980)
%1982 : Tensor = onnx::Sqrt(%1981)
%1983 : FloatTensor = onnx::Div(%1975, %1982)
%1984 : FloatTensor = onnx::Mul(%1983, %decoder.layers.10.norm2.weight)
%1985 : Float(58:512, 1:512, 512:1) = onnx::Add(%1984, %decoder.layers.10.norm2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%1986 : Tensor = onnx::Shape(%1985)
%1987 : Tensor = onnx::Constant[value={0}]()
%1988 : Long() = onnx::Gather[axis=0](%1986, %1987) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1989 : Tensor = onnx::Shape(%1985)
%1990 : Tensor = onnx::Constant[value={1}]()
%1991 : Long() = onnx::Gather[axis=0](%1989, %1990) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1992 : Tensor = onnx::Shape(%1985)
%1993 : Tensor = onnx::Constant[value={2}]()
%1994 : Long() = onnx::Gather[axis=0](%1992, %1993) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%1995 : Long() = onnx::Constant[value={2}]()
%1996 : LongTensor = onnx::Div(%1994, %1995)
%1997 : Tensor = onnx::Cast[to=7](%1996)
%1998 : Long() = onnx::Cast[to=7](%1997) # /opt/conda/lib/python3.7/site-packages/torch/tensor.py:424:0
%2000 : Float(58:1536, 1:1536, 1536:1) = onnx::MatMul(%1985, %2551) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%2001 : Float(58:1536, 1:1536, 1536:1) = onnx::Add(%2000, %decoder.layers.11.self_attn.in_proj_bias)
%2002 : Float(58:1536, 1:1536, 512:1), %2003 : Float(58:512, 1:512, 512:1), %2004 : Float(58:512, 1:512, 512:1) = onnx::Split[axis=-1, split=[512, 512, 512]](%2001) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%2005 : Float() = onnx::Constant[value={0.0625}]()
%2006 : Float(58:512, 1:512, 512:1) = onnx::Mul(%2002, %2005) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%2007 : Long() = onnx::Constant[value={2}]()
%2008 : Long() = onnx::Mul(%1991, %2007)
%2009 : Tensor = onnx::Unsqueeze[axes=[0]](%1988)
%2010 : Tensor = onnx::Unsqueeze[axes=[0]](%2008)
%2011 : Tensor = onnx::Unsqueeze[axes=[0]](%1998)
%2012 : Tensor = onnx::Concat[axis=0](%2009, %2010, %2011)
%2013 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%2006, %2012) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%2014 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%2013) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%2015 : Long() = onnx::Constant[value={2}]()
%2016 : Long() = onnx::Mul(%1991, %2015)
%2019 : Tensor = onnx::Unsqueeze[axes=[0]](%2016)
%2020 : Tensor = onnx::Unsqueeze[axes=[0]](%1998)
%2021 : Tensor = onnx::Concat[axis=0](%2552, %2019, %2020)
%2022 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%2003, %2021) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4079:0
%2023 : Long() = onnx::Constant[value={2}]()
%2024 : Long() = onnx::Mul(%1991, %2023)
%2027 : Tensor = onnx::Unsqueeze[axes=[0]](%2024)
%2028 : Tensor = onnx::Unsqueeze[axes=[0]](%1998)
%2029 : Tensor = onnx::Concat[axis=0](%2553, %2027, %2028)
%2030 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%2004, %2029) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%2031 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%2030) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%2032 : Float(2:256, 256:1, 58:512) = onnx::Transpose[perm=[1, 2, 0]](%2022) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%2033 : Float(2:3364, 58:58, 58:1) = onnx::MatMul(%2014, %2032) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%2034 : Float(2:3364, 58:58, 58:1) = onnx::Softmax[axis=2](%2033) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%2035 : Float(2:14848, 58:256, 256:1) = onnx::MatMul(%2034, %2031) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4130:0
%2036 : Float(58:512, 2:256, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%2035) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%2037 : Tensor = onnx::Unsqueeze[axes=[0]](%1988)
%2038 : Tensor = onnx::Unsqueeze[axes=[0]](%1991)
%2039 : Tensor = onnx::Unsqueeze[axes=[0]](%1994)
%2040 : Tensor = onnx::Concat[axis=0](%2037, %2038, %2039)
%2041 : Float(58:512, 1:512, 512:1) = onnx::Reshape(%2036, %2040) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%2043 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%2041, %2554) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%2044 : Float(58:512, 1:512, 512:1) = onnx::Add(%2043, %decoder.layers.11.self_attn.out_proj.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%2045 : Float(58:512, 1:512, 512:1) = onnx::Add(%1985, %2044) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:295:0
%2047 : Tensor = onnx::ReduceMean[axes=[-1]](%2045)
%2048 : FloatTensor = onnx::Sub(%2045, %2047)
%2049 : Tensor = onnx::Cast[to=1](%2048)
%2051 : Tensor = onnx::Pow(%2049, %2555)
%2052 : Tensor = onnx::ReduceMean[axes=[-1]](%2051)
%2053 : Float() = onnx::Constant[value={1e-05}]()
%2054 : FloatTensor = onnx::Add(%2052, %2053)
%2055 : Tensor = onnx::Sqrt(%2054)
%2056 : FloatTensor = onnx::Div(%2048, %2055)
%2057 : FloatTensor = onnx::Mul(%2056, %decoder.layers.11.norm1.weight)
%2058 : Float(58:512, 1:512, 512:1) = onnx::Add(%2057, %decoder.layers.11.norm1.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%2060 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%2058, %2556) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%2061 : Float(58:512, 1:512, 512:1) = onnx::Add(%2060, %decoder.layers.11.linear1.bias)
%2062 : Float(58:512, 1:512, 512:1) = onnx::Relu(%2061) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%2064 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%2062, %2557) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%2065 : Float(58:512, 1:512, 512:1) = onnx::Add(%2064, %decoder.layers.11.linear2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%2066 : Float(58:512, 1:512, 512:1) = onnx::Add(%2058, %2065) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:298:0
%2068 : Tensor = onnx::ReduceMean[axes=[-1]](%2066)
%2069 : FloatTensor = onnx::Sub(%2066, %2068)
%2070 : Tensor = onnx::Cast[to=1](%2069)
%2072 : Tensor = onnx::Pow(%2070, %2558)
%2073 : Tensor = onnx::ReduceMean[axes=[-1]](%2072)
%2074 : Float() = onnx::Constant[value={1e-05}]()
%2075 : FloatTensor = onnx::Add(%2073, %2074)
%2076 : Tensor = onnx::Sqrt(%2075)
%2077 : FloatTensor = onnx::Div(%2069, %2076)
%2078 : FloatTensor = onnx::Mul(%2077, %decoder.layers.11.norm2.weight)
%2079 : Float(58:512, 1:512, 512:1) = onnx::Add(%2078, %decoder.layers.11.norm2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%2080 : Tensor = onnx::Shape(%2079)
%2081 : Tensor = onnx::Constant[value={0}]()
%2082 : Long() = onnx::Gather[axis=0](%2080, %2081) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%2083 : Tensor = onnx::Shape(%2079)
%2084 : Tensor = onnx::Constant[value={1}]()
%2085 : Long() = onnx::Gather[axis=0](%2083, %2084) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%2086 : Tensor = onnx::Shape(%2079)
%2087 : Tensor = onnx::Constant[value={2}]()
%2088 : Long() = onnx::Gather[axis=0](%2086, %2087) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%2089 : Long() = onnx::Constant[value={2}]()
%2090 : LongTensor = onnx::Div(%2088, %2089)
%2091 : Tensor = onnx::Cast[to=7](%2090)
%2092 : Long() = onnx::Cast[to=7](%2091) # /opt/conda/lib/python3.7/site-packages/torch/tensor.py:424:0
%2094 : Float(58:1536, 1:1536, 1536:1) = onnx::MatMul(%2079, %2559) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%2095 : Float(58:1536, 1:1536, 1536:1) = onnx::Add(%2094, %decoder.layers.12.self_attn.in_proj_bias)
%2096 : Float(58:1536, 1:1536, 512:1), %2097 : Float(58:512, 1:512, 512:1), %2098 : Float(58:512, 1:512, 512:1) = onnx::Split[axis=-1, split=[512, 512, 512]](%2095) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%2099 : Float() = onnx::Constant[value={0.0625}]()
%2100 : Float(58:512, 1:512, 512:1) = onnx::Mul(%2096, %2099) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%2101 : Long() = onnx::Constant[value={2}]()
%2102 : Long() = onnx::Mul(%2085, %2101)
%2103 : Tensor = onnx::Unsqueeze[axes=[0]](%2082)
%2104 : Tensor = onnx::Unsqueeze[axes=[0]](%2102)
%2105 : Tensor = onnx::Unsqueeze[axes=[0]](%2092)
%2106 : Tensor = onnx::Concat[axis=0](%2103, %2104, %2105)
%2107 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%2100, %2106) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%2108 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%2107) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%2109 : Long() = onnx::Constant[value={2}]()
%2110 : Long() = onnx::Mul(%2085, %2109)
%2113 : Tensor = onnx::Unsqueeze[axes=[0]](%2110)
%2114 : Tensor = onnx::Unsqueeze[axes=[0]](%2092)
%2115 : Tensor = onnx::Concat[axis=0](%2560, %2113, %2114)
%2116 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%2097, %2115) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4079:0
%2117 : Long() = onnx::Constant[value={2}]()
%2118 : Long() = onnx::Mul(%2085, %2117)
%2121 : Tensor = onnx::Unsqueeze[axes=[0]](%2118)
%2122 : Tensor = onnx::Unsqueeze[axes=[0]](%2092)
%2123 : Tensor = onnx::Concat[axis=0](%2561, %2121, %2122)
%2124 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%2098, %2123) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%2125 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%2124) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%2126 : Float(2:256, 256:1, 58:512) = onnx::Transpose[perm=[1, 2, 0]](%2116) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%2127 : Float(2:3364, 58:58, 58:1) = onnx::MatMul(%2108, %2126) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%2128 : Float(2:3364, 58:58, 58:1) = onnx::Softmax[axis=2](%2127) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%2129 : Float(2:14848, 58:256, 256:1) = onnx::MatMul(%2128, %2125) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4130:0
%2130 : Float(58:512, 2:256, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%2129) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%2131 : Tensor = onnx::Unsqueeze[axes=[0]](%2082)
%2132 : Tensor = onnx::Unsqueeze[axes=[0]](%2085)
%2133 : Tensor = onnx::Unsqueeze[axes=[0]](%2088)
%2134 : Tensor = onnx::Concat[axis=0](%2131, %2132, %2133)
%2135 : Float(58:512, 1:512, 512:1) = onnx::Reshape(%2130, %2134) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%2137 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%2135, %2562) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%2138 : Float(58:512, 1:512, 512:1) = onnx::Add(%2137, %decoder.layers.12.self_attn.out_proj.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%2139 : Float(58:512, 1:512, 512:1) = onnx::Add(%2079, %2138) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:295:0
%2141 : Tensor = onnx::ReduceMean[axes=[-1]](%2139)
%2142 : FloatTensor = onnx::Sub(%2139, %2141)
%2143 : Tensor = onnx::Cast[to=1](%2142)
%2145 : Tensor = onnx::Pow(%2143, %2563)
%2146 : Tensor = onnx::ReduceMean[axes=[-1]](%2145)
%2147 : Float() = onnx::Constant[value={1e-05}]()
%2148 : FloatTensor = onnx::Add(%2146, %2147)
%2149 : Tensor = onnx::Sqrt(%2148)
%2150 : FloatTensor = onnx::Div(%2142, %2149)
%2151 : FloatTensor = onnx::Mul(%2150, %decoder.layers.12.norm1.weight)
%2152 : Float(58:512, 1:512, 512:1) = onnx::Add(%2151, %decoder.layers.12.norm1.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%2154 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%2152, %2564) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%2155 : Float(58:512, 1:512, 512:1) = onnx::Add(%2154, %decoder.layers.12.linear1.bias)
%2156 : Float(58:512, 1:512, 512:1) = onnx::Relu(%2155) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%2158 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%2156, %2565) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%2159 : Float(58:512, 1:512, 512:1) = onnx::Add(%2158, %decoder.layers.12.linear2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%2160 : Float(58:512, 1:512, 512:1) = onnx::Add(%2152, %2159) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:298:0
%2162 : Tensor = onnx::ReduceMean[axes=[-1]](%2160)
%2163 : FloatTensor = onnx::Sub(%2160, %2162)
%2164 : Tensor = onnx::Cast[to=1](%2163)
%2166 : Tensor = onnx::Pow(%2164, %2566)
%2167 : Tensor = onnx::ReduceMean[axes=[-1]](%2166)
%2168 : Float() = onnx::Constant[value={1e-05}]()
%2169 : FloatTensor = onnx::Add(%2167, %2168)
%2170 : Tensor = onnx::Sqrt(%2169)
%2171 : FloatTensor = onnx::Div(%2163, %2170)
%2172 : FloatTensor = onnx::Mul(%2171, %decoder.layers.12.norm2.weight)
%2173 : Float(58:512, 1:512, 512:1) = onnx::Add(%2172, %decoder.layers.12.norm2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%2174 : Tensor = onnx::Shape(%2173)
%2175 : Tensor = onnx::Constant[value={0}]()
%2176 : Long() = onnx::Gather[axis=0](%2174, %2175) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%2177 : Tensor = onnx::Shape(%2173)
%2178 : Tensor = onnx::Constant[value={1}]()
%2179 : Long() = onnx::Gather[axis=0](%2177, %2178) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%2180 : Tensor = onnx::Shape(%2173)
%2181 : Tensor = onnx::Constant[value={2}]()
%2182 : Long() = onnx::Gather[axis=0](%2180, %2181) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%2183 : Long() = onnx::Constant[value={2}]()
%2184 : LongTensor = onnx::Div(%2182, %2183)
%2185 : Tensor = onnx::Cast[to=7](%2184)
%2186 : Long() = onnx::Cast[to=7](%2185) # /opt/conda/lib/python3.7/site-packages/torch/tensor.py:424:0
%2188 : Float(58:1536, 1:1536, 1536:1) = onnx::MatMul(%2173, %2567) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%2189 : Float(58:1536, 1:1536, 1536:1) = onnx::Add(%2188, %decoder.layers.13.self_attn.in_proj_bias)
%2190 : Float(58:1536, 1:1536, 512:1), %2191 : Float(58:512, 1:512, 512:1), %2192 : Float(58:512, 1:512, 512:1) = onnx::Split[axis=-1, split=[512, 512, 512]](%2189) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%2193 : Float() = onnx::Constant[value={0.0625}]()
%2194 : Float(58:512, 1:512, 512:1) = onnx::Mul(%2190, %2193) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%2195 : Long() = onnx::Constant[value={2}]()
%2196 : Long() = onnx::Mul(%2179, %2195)
%2197 : Tensor = onnx::Unsqueeze[axes=[0]](%2176)
%2198 : Tensor = onnx::Unsqueeze[axes=[0]](%2196)
%2199 : Tensor = onnx::Unsqueeze[axes=[0]](%2186)
%2200 : Tensor = onnx::Concat[axis=0](%2197, %2198, %2199)
%2201 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%2194, %2200) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%2202 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%2201) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%2203 : Long() = onnx::Constant[value={2}]()
%2204 : Long() = onnx::Mul(%2179, %2203)
%2207 : Tensor = onnx::Unsqueeze[axes=[0]](%2204)
%2208 : Tensor = onnx::Unsqueeze[axes=[0]](%2186)
%2209 : Tensor = onnx::Concat[axis=0](%2568, %2207, %2208)
%2210 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%2191, %2209) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4079:0
%2211 : Long() = onnx::Constant[value={2}]()
%2212 : Long() = onnx::Mul(%2179, %2211)
%2215 : Tensor = onnx::Unsqueeze[axes=[0]](%2212)
%2216 : Tensor = onnx::Unsqueeze[axes=[0]](%2186)
%2217 : Tensor = onnx::Concat[axis=0](%2569, %2215, %2216)
%2218 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%2192, %2217) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%2219 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%2218) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%2220 : Float(2:256, 256:1, 58:512) = onnx::Transpose[perm=[1, 2, 0]](%2210) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%2221 : Float(2:3364, 58:58, 58:1) = onnx::MatMul(%2202, %2220) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%2222 : Float(2:3364, 58:58, 58:1) = onnx::Softmax[axis=2](%2221) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%2223 : Float(2:14848, 58:256, 256:1) = onnx::MatMul(%2222, %2219) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4130:0
%2224 : Float(58:512, 2:256, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%2223) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%2225 : Tensor = onnx::Unsqueeze[axes=[0]](%2176)
%2226 : Tensor = onnx::Unsqueeze[axes=[0]](%2179)
%2227 : Tensor = onnx::Unsqueeze[axes=[0]](%2182)
%2228 : Tensor = onnx::Concat[axis=0](%2225, %2226, %2227)
%2229 : Float(58:512, 1:512, 512:1) = onnx::Reshape(%2224, %2228) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%2231 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%2229, %2570) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%2232 : Float(58:512, 1:512, 512:1) = onnx::Add(%2231, %decoder.layers.13.self_attn.out_proj.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%2233 : Float(58:512, 1:512, 512:1) = onnx::Add(%2173, %2232) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:295:0
%2235 : Tensor = onnx::ReduceMean[axes=[-1]](%2233)
%2236 : FloatTensor = onnx::Sub(%2233, %2235)
%2237 : Tensor = onnx::Cast[to=1](%2236)
%2239 : Tensor = onnx::Pow(%2237, %2571)
%2240 : Tensor = onnx::ReduceMean[axes=[-1]](%2239)
%2241 : Float() = onnx::Constant[value={1e-05}]()
%2242 : FloatTensor = onnx::Add(%2240, %2241)
%2243 : Tensor = onnx::Sqrt(%2242)
%2244 : FloatTensor = onnx::Div(%2236, %2243)
%2245 : FloatTensor = onnx::Mul(%2244, %decoder.layers.13.norm1.weight)
%2246 : Float(58:512, 1:512, 512:1) = onnx::Add(%2245, %decoder.layers.13.norm1.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%2248 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%2246, %2572) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%2249 : Float(58:512, 1:512, 512:1) = onnx::Add(%2248, %decoder.layers.13.linear1.bias)
%2250 : Float(58:512, 1:512, 512:1) = onnx::Relu(%2249) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%2252 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%2250, %2573) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%2253 : Float(58:512, 1:512, 512:1) = onnx::Add(%2252, %decoder.layers.13.linear2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%2254 : Float(58:512, 1:512, 512:1) = onnx::Add(%2246, %2253) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:298:0
%2256 : Tensor = onnx::ReduceMean[axes=[-1]](%2254)
%2257 : FloatTensor = onnx::Sub(%2254, %2256)
%2258 : Tensor = onnx::Cast[to=1](%2257)
%2260 : Tensor = onnx::Pow(%2258, %2574)
%2261 : Tensor = onnx::ReduceMean[axes=[-1]](%2260)
%2262 : Float() = onnx::Constant[value={1e-05}]()
%2263 : FloatTensor = onnx::Add(%2261, %2262)
%2264 : Tensor = onnx::Sqrt(%2263)
%2265 : FloatTensor = onnx::Div(%2257, %2264)
%2266 : FloatTensor = onnx::Mul(%2265, %decoder.layers.13.norm2.weight)
%2267 : Float(58:512, 1:512, 512:1) = onnx::Add(%2266, %decoder.layers.13.norm2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%2268 : Tensor = onnx::Shape(%2267)
%2269 : Tensor = onnx::Constant[value={0}]()
%2270 : Long() = onnx::Gather[axis=0](%2268, %2269) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%2271 : Tensor = onnx::Shape(%2267)
%2272 : Tensor = onnx::Constant[value={1}]()
%2273 : Long() = onnx::Gather[axis=0](%2271, %2272) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%2274 : Tensor = onnx::Shape(%2267)
%2275 : Tensor = onnx::Constant[value={2}]()
%2276 : Long() = onnx::Gather[axis=0](%2274, %2275) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%2277 : Long() = onnx::Constant[value={2}]()
%2278 : LongTensor = onnx::Div(%2276, %2277)
%2279 : Tensor = onnx::Cast[to=7](%2278)
%2280 : Long() = onnx::Cast[to=7](%2279) # /opt/conda/lib/python3.7/site-packages/torch/tensor.py:424:0
%2282 : Float(58:1536, 1:1536, 1536:1) = onnx::MatMul(%2267, %2575) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%2283 : Float(58:1536, 1:1536, 1536:1) = onnx::Add(%2282, %decoder.layers.14.self_attn.in_proj_bias)
%2284 : Float(58:1536, 1:1536, 512:1), %2285 : Float(58:512, 1:512, 512:1), %2286 : Float(58:512, 1:512, 512:1) = onnx::Split[axis=-1, split=[512, 512, 512]](%2283) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%2287 : Float() = onnx::Constant[value={0.0625}]()
%2288 : Float(58:512, 1:512, 512:1) = onnx::Mul(%2284, %2287) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%2289 : Long() = onnx::Constant[value={2}]()
%2290 : Long() = onnx::Mul(%2273, %2289)
%2291 : Tensor = onnx::Unsqueeze[axes=[0]](%2270)
%2292 : Tensor = onnx::Unsqueeze[axes=[0]](%2290)
%2293 : Tensor = onnx::Unsqueeze[axes=[0]](%2280)
%2294 : Tensor = onnx::Concat[axis=0](%2291, %2292, %2293)
%2295 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%2288, %2294) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%2296 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%2295) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%2297 : Long() = onnx::Constant[value={2}]()
%2298 : Long() = onnx::Mul(%2273, %2297)
%2301 : Tensor = onnx::Unsqueeze[axes=[0]](%2298)
%2302 : Tensor = onnx::Unsqueeze[axes=[0]](%2280)
%2303 : Tensor = onnx::Concat[axis=0](%2576, %2301, %2302)
%2304 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%2285, %2303) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4079:0
%2305 : Long() = onnx::Constant[value={2}]()
%2306 : Long() = onnx::Mul(%2273, %2305)
%2309 : Tensor = onnx::Unsqueeze[axes=[0]](%2306)
%2310 : Tensor = onnx::Unsqueeze[axes=[0]](%2280)
%2311 : Tensor = onnx::Concat[axis=0](%2577, %2309, %2310)
%2312 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%2286, %2311) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%2313 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%2312) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%2314 : Float(2:256, 256:1, 58:512) = onnx::Transpose[perm=[1, 2, 0]](%2304) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%2315 : Float(2:3364, 58:58, 58:1) = onnx::MatMul(%2296, %2314) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%2316 : Float(2:3364, 58:58, 58:1) = onnx::Softmax[axis=2](%2315) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%2317 : Float(2:14848, 58:256, 256:1) = onnx::MatMul(%2316, %2313) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4130:0
%2318 : Float(58:512, 2:256, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%2317) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%2319 : Tensor = onnx::Unsqueeze[axes=[0]](%2270)
%2320 : Tensor = onnx::Unsqueeze[axes=[0]](%2273)
%2321 : Tensor = onnx::Unsqueeze[axes=[0]](%2276)
%2322 : Tensor = onnx::Concat[axis=0](%2319, %2320, %2321)
%2323 : Float(58:512, 1:512, 512:1) = onnx::Reshape(%2318, %2322) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%2325 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%2323, %2578) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%2326 : Float(58:512, 1:512, 512:1) = onnx::Add(%2325, %decoder.layers.14.self_attn.out_proj.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%2327 : Float(58:512, 1:512, 512:1) = onnx::Add(%2267, %2326) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:295:0
%2329 : Tensor = onnx::ReduceMean[axes=[-1]](%2327)
%2330 : FloatTensor = onnx::Sub(%2327, %2329)
%2331 : Tensor = onnx::Cast[to=1](%2330)
%2333 : Tensor = onnx::Pow(%2331, %2579)
%2334 : Tensor = onnx::ReduceMean[axes=[-1]](%2333)
%2335 : Float() = onnx::Constant[value={1e-05}]()
%2336 : FloatTensor = onnx::Add(%2334, %2335)
%2337 : Tensor = onnx::Sqrt(%2336)
%2338 : FloatTensor = onnx::Div(%2330, %2337)
%2339 : FloatTensor = onnx::Mul(%2338, %decoder.layers.14.norm1.weight)
%2340 : Float(58:512, 1:512, 512:1) = onnx::Add(%2339, %decoder.layers.14.norm1.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%2342 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%2340, %2580) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%2343 : Float(58:512, 1:512, 512:1) = onnx::Add(%2342, %decoder.layers.14.linear1.bias)
%2344 : Float(58:512, 1:512, 512:1) = onnx::Relu(%2343) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%2346 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%2344, %2581) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%2347 : Float(58:512, 1:512, 512:1) = onnx::Add(%2346, %decoder.layers.14.linear2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%2348 : Float(58:512, 1:512, 512:1) = onnx::Add(%2340, %2347) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:298:0
%2350 : Tensor = onnx::ReduceMean[axes=[-1]](%2348)
%2351 : FloatTensor = onnx::Sub(%2348, %2350)
%2352 : Tensor = onnx::Cast[to=1](%2351)
%2354 : Tensor = onnx::Pow(%2352, %2582)
%2355 : Tensor = onnx::ReduceMean[axes=[-1]](%2354)
%2356 : Float() = onnx::Constant[value={1e-05}]()
%2357 : FloatTensor = onnx::Add(%2355, %2356)
%2358 : Tensor = onnx::Sqrt(%2357)
%2359 : FloatTensor = onnx::Div(%2351, %2358)
%2360 : FloatTensor = onnx::Mul(%2359, %decoder.layers.14.norm2.weight)
%2361 : Float(58:512, 1:512, 512:1) = onnx::Add(%2360, %decoder.layers.14.norm2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%2362 : Tensor = onnx::Shape(%2361)
%2363 : Tensor = onnx::Constant[value={0}]()
%2364 : Long() = onnx::Gather[axis=0](%2362, %2363) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%2365 : Tensor = onnx::Shape(%2361)
%2366 : Tensor = onnx::Constant[value={1}]()
%2367 : Long() = onnx::Gather[axis=0](%2365, %2366) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%2368 : Tensor = onnx::Shape(%2361)
%2369 : Tensor = onnx::Constant[value={2}]()
%2370 : Long() = onnx::Gather[axis=0](%2368, %2369) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3948:0
%2371 : Long() = onnx::Constant[value={2}]()
%2372 : LongTensor = onnx::Div(%2370, %2371)
%2373 : Tensor = onnx::Cast[to=7](%2372)
%2374 : Long() = onnx::Cast[to=7](%2373) # /opt/conda/lib/python3.7/site-packages/torch/tensor.py:424:0
%2376 : Float(58:1536, 1:1536, 1536:1) = onnx::MatMul(%2361, %2583) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%2377 : Float(58:1536, 1:1536, 1536:1) = onnx::Add(%2376, %decoder.layers.15.self_attn.in_proj_bias)
%2378 : Float(58:1536, 1:1536, 512:1), %2379 : Float(58:512, 1:512, 512:1), %2380 : Float(58:512, 1:512, 512:1) = onnx::Split[axis=-1, split=[512, 512, 512]](%2377) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%2381 : Float() = onnx::Constant[value={0.0625}]()
%2382 : Float(58:512, 1:512, 512:1) = onnx::Mul(%2378, %2381) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%2383 : Long() = onnx::Constant[value={2}]()
%2384 : Long() = onnx::Mul(%2367, %2383)
%2385 : Tensor = onnx::Unsqueeze[axes=[0]](%2364)
%2386 : Tensor = onnx::Unsqueeze[axes=[0]](%2384)
%2387 : Tensor = onnx::Unsqueeze[axes=[0]](%2374)
%2388 : Tensor = onnx::Concat[axis=0](%2385, %2386, %2387)
%2389 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%2382, %2388) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%2390 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%2389) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4077:0
%2391 : Long() = onnx::Constant[value={2}]()
%2392 : Long() = onnx::Mul(%2367, %2391)
%2395 : Tensor = onnx::Unsqueeze[axes=[0]](%2392)
%2396 : Tensor = onnx::Unsqueeze[axes=[0]](%2374)
%2397 : Tensor = onnx::Concat[axis=0](%2584, %2395, %2396)
%2398 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%2379, %2397) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4079:0
%2399 : Long() = onnx::Constant[value={2}]()
%2400 : Long() = onnx::Mul(%2367, %2399)
%2403 : Tensor = onnx::Unsqueeze[axes=[0]](%2400)
%2404 : Tensor = onnx::Unsqueeze[axes=[0]](%2374)
%2405 : Tensor = onnx::Concat[axis=0](%2585, %2403, %2404)
%2406 : Float(58:512, 2:256, 256:1) = onnx::Reshape(%2380, %2405) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%2407 : Float(2:256, 58:512, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%2406) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4081:0
%2408 : Float(2:256, 256:1, 58:512) = onnx::Transpose[perm=[1, 2, 0]](%2398) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%2409 : Float(2:3364, 58:58, 58:1) = onnx::MatMul(%2390, %2408) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4108:0
%2410 : Float(2:3364, 58:58, 58:1) = onnx::Softmax[axis=2](%2409) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%2411 : Float(2:14848, 58:256, 256:1) = onnx::MatMul(%2410, %2407) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4130:0
%2412 : Float(58:512, 2:256, 256:1) = onnx::Transpose[perm=[1, 0, 2]](%2411) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%2413 : Tensor = onnx::Unsqueeze[axes=[0]](%2364)
%2414 : Tensor = onnx::Unsqueeze[axes=[0]](%2367)
%2415 : Tensor = onnx::Unsqueeze[axes=[0]](%2370)
%2416 : Tensor = onnx::Concat[axis=0](%2413, %2414, %2415)
%2417 : Float(58:512, 1:512, 512:1) = onnx::Reshape(%2412, %2416) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:4132:0
%2419 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%2417, %2586) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%2420 : Float(58:512, 1:512, 512:1) = onnx::Add(%2419, %decoder.layers.15.self_attn.out_proj.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%2421 : Float(58:512, 1:512, 512:1) = onnx::Add(%2361, %2420) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:295:0
%2423 : Tensor = onnx::ReduceMean[axes=[-1]](%2421)
%2424 : FloatTensor = onnx::Sub(%2421, %2423)
%2425 : Tensor = onnx::Cast[to=1](%2424)
%2427 : Tensor = onnx::Pow(%2425, %2587)
%2428 : Tensor = onnx::ReduceMean[axes=[-1]](%2427)
%2429 : Float() = onnx::Constant[value={1e-05}]()
%2430 : FloatTensor = onnx::Add(%2428, %2429)
%2431 : Tensor = onnx::Sqrt(%2430)
%2432 : FloatTensor = onnx::Div(%2424, %2431)
%2433 : FloatTensor = onnx::Mul(%2432, %decoder.layers.15.norm1.weight)
%2434 : Float(58:512, 1:512, 512:1) = onnx::Add(%2433, %decoder.layers.15.norm1.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%2436 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%2434, %2588) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%2437 : Float(58:512, 1:512, 512:1) = onnx::Add(%2436, %decoder.layers.15.linear1.bias)
%2438 : Float(58:512, 1:512, 512:1) = onnx::Relu(%2437) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%2440 : Float(58:512, 1:512, 512:1) = onnx::MatMul(%2438, %2589) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1676:0
%2441 : Float(58:512, 1:512, 512:1) = onnx::Add(%2440, %decoder.layers.15.linear2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:973:0
%2442 : Float(58:512, 1:512, 512:1) = onnx::Add(%2434, %2441) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/transformer.py:298:0
%2444 : Tensor = onnx::ReduceMean[axes=[-1]](%2442)
%2445 : FloatTensor = onnx::Sub(%2442, %2444)
%2446 : Tensor = onnx::Cast[to=1](%2445)
%2448 : Tensor = onnx::Pow(%2446, %2590)
%2449 : Tensor = onnx::ReduceMean[axes=[-1]](%2448)
%2450 : Float() = onnx::Constant[value={1e-05}]()
%2451 : FloatTensor = onnx::Add(%2449, %2450)
%2452 : Tensor = onnx::Sqrt(%2451)
%2453 : FloatTensor = onnx::Div(%2445, %2452)
%2454 : FloatTensor = onnx::Mul(%2453, %decoder.layers.15.norm2.weight)
%2455 : Float(58:512, 1:512, 512:1) = onnx::Add(%2454, %decoder.layers.15.norm2.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:2049:0
%2456 : Float(1:29696, 512:58, 58:1) = onnx::Transpose[perm=[1, 2, 0]](%2455) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model.py:668:0
%2457 : Float(1:57942, 999:58, 58:1) = onnx::Conv[dilations=[1], group=1, kernel_shape=[1], pads=[0, 0], strides=[1]](%2456, %fc.weight, %fc.bias) # /opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py:257:0
%2458 : Float(1:57942, 58:999, 999:1) = onnx::Transpose[perm=[0, 2, 1]](%2457) # /home/keras/notebook/nvme/aveysov/open_stt_pretrained_deploy/stt_pretrained/models/model.py:671:0
%output : Float(1:57942, 58:999, 999:1) = onnx::Softmax[axis=2](%2458) # /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1498:0
return (%output)

Question

Looks like int64 storage of some constants is an artefact of ONNX and PyTorch export.
I have seen that in other ONNX-related projects this issue was widely addressed.
Can this be done here as well?
Or does anyone have a recipe to export models differently?

Alternative Solutions

Transform to TensorFlow, then to tf.js.
Looks like an inferior option, just because you have to transform 3 times instead of one.
I will also raise a similar issue on PyTorch forums.

The text was updated successfully, but these errors were encountered:

KnurpsBram · 2020-10-26T08:44:21Z

I too would love to see int64 supported in onnx.js. It would make exporting a PyTorch model and running it in JS much more straightforward. I had originally posted a question on pytorch-to-onnx conversion on the PyTorch forum:
https://discuss.pytorch.org/t/pytorch-onnx-javascript-model-with-reflectionpad-and-convtranspose/98417

serg06 · 2021-06-08T18:11:17Z

Still praying for this.

Why does it have to be so hard to type cast some weights from int64 to int32?

snakers4 mentioned this issue Oct 7, 2020

ONNX Export - Dealing with INT64 Constants and Slices pytorch/pytorch#45957

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature Request - Dealing with int64 in the exported ONNX model #233

Feature Request - Dealing with int64 in the exported ONNX model #233

snakers4 commented Oct 7, 2020

KnurpsBram commented Oct 26, 2020

serg06 commented Jun 8, 2021

Feature Request - Dealing with int64 in the exported ONNX model #233

Feature Request - Dealing with int64 in the exported ONNX model #233

Comments

snakers4 commented Oct 7, 2020

Background

Model Export

Problem

Question

Alternative Solutions

KnurpsBram commented Oct 26, 2020

serg06 commented Jun 8, 2021