[Experimental / WIP] Generate a miniscule model for each OP only, retain the results of test inference, and propagate to all OPs. STEP.1,2

[PINTO0309]

1. Content and background

• [Experimental / WIP] Generate a miniscule model for each OP only, retain the results of test inference, and propagate to all OPs.
  • First step
    • Softmax
    • Sub
    • ReduceMax
    • Reshape
      In the figure below, the output from the first Reshape only model is propagated to ReduceMax and Sub. Next, a ReduceMax only model is generated and test inference is performed based on the inference values propagated from the Reshape model.
    • In other words, it generates models for only one OP each and stores all test inference results for each OP alone.
    • By using a method that sequentially stores inference results in a model with only one OP, it becomes easy to backward from the point where an inference accuracy error is detected and rerun the inference test from the OP where the problem is assumed to have occurred.
      
  • Second step
    • To attempt to speed up the inference checks in ddnm.onnx, the OPs in the following list will be addressed.
    • Eliminate Softmax's redundant multiple inference accuracy correction process.
    • Implement on a trial basis in the following OPs for verification.
      • Add
      • Concat
      • Conv
      • Cos
      • Gemm
      • InstanceNormalization
      • MatMul
      • Mul
      • Div
      • Mod
      • ReduceMax
      • Reshape
      • Resize
      • Sigmoid
      • Sin
      • Softmax
      • Sub
      • Transpose
      • Unsqueeze

      ssc4onnx -if ddnm.onnx
      ┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
      ┃ OP Type                ┃ OPs        ┃
      ┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
      │ Add                    │ 141        │
      │ Concat                 │ 19         │
      │ Conv                   │ 120        │
      │ Cos                    │ 1          │
      │ Gemm                   │ 34         │
      │ InstanceNormalization  │ 71         │
      │ MatMul                 │ 12         │
      │ Mul                    │ 145        │
      │ Reshape                │ 166        │
      │ Resize                 │ 5          │
      │ Sigmoid                │ 67         │
      │ Sin                    │ 1          │
      │ Softmax                │ 6          │
      │ Transpose              │ 12         │
      │ Unsqueeze              │ 65         │
      │ ---------------------- │ ---------- │
      │ Total number of OPs    │ 865        │
      │ ====================== │ ========== │
      │ Model Size             │ 433.8MiB   │
      └────────────────────────┴────────────┘
      

  • Third step - Will be separated into separate pull requests
    • Add a process to perform the calculation of the maximum absolute error on the inference results of all OPs.
  • Fourth step - Will be separated into separate pull requests
    • Implement propagation of inference results for all operations other than the above.
  • Fifth step - Will be separated into separate pull requests
    • Instead of a processing format for inferring TensorFlow models, primitive operations are replaced with Numpy operations to make the process as fast as possible.
  • Sixth step - Will be separated into separate pull requests
    • Seek ways to eliminate accuracy errors without creating JSON files.
    • [DDNM] Support additional parameters to onnxsim #175

2. Summary of corrections

3. Before/After (If there is an operating log that can be used as a reference)

4. Issue number (only if there is a related issue)

Implementation of strict mode #145
[DDNM] Support additional parameters to onnxsim #175