Skip to content

[BUG] The pass "eliminate_slice_after_shape" causes great output mismatch #263

@C1ri007

Description

@C1ri007

The pass "eliminate_slice_after_shape" causes output mismatch

Issue
Running the single pass eliminate_slice_after_shape with onnxoptimizer 0.4.2 may change model outputs. This folder provides minimal repro cases and harnesses to compare original vs optimized models using stored oracle inputs.

Environment

  • Ubuntu 20.04
  • Python 3.10
  • onnx==1.19.0
  • onnxruntime==1.19.2
  • onnxoptimizer==0.4.2 (latest)

Repro steps (run from this folder)

eliminate_slice_after_shape_repro.tar.gz

  1. Create a Python 3.10 virtual environment and install dependencies:
python -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt
  1. Optimize a case with only eliminate_slice_after_shape (writes model.opt.onnx next to model.onnx):
python optimize_model.py --case ./case_02084_seed36754215
  1. Differential test original vs optimized outputs using stored oracle inputs :
python diff_test.py --case ./case_02084_seed36754215
  1. Repeat Step 1 ~ 3 until all the models involved are tested.

Observed results

  • Mismatches:
    • case_03361_seed46866778: output v52_0 drifts (max_abs=6.770e+00, max_rel=1.417e+00).
    • case_05414_seed63124485: output v26_0 drifts (max_abs=1.067e+19, max_rel=2.512e+00).
    • case_05714_seed65500185: output v10_0 drifts (max_abs=1.153e+00, max_rel=1.005e+00).

Expected
eliminate_slice_after_shape should be semantics-preserving. Applying this pass alone should not change any output values.

Differential Test Output Details

  1. case_03361_seed46866778
Case: case_03361_seed46866778
Mismatched outputs (optimized vs reference):
	v52_0 (output[4]): max_abs=6.770e+00, max_rel=1.417e+00, shape=(34, 34, 1, 22)
Overall: max_abs=6.770e+00, max_rel=1.417e+00
  1. case_05414_seed63124485
Case: case_05414_seed63124485
Mismatched outputs (optimized vs reference):
	v26_0 (output[10]): max_abs=1.067e+19, max_rel=2.512e+00, shape=(33, 52)
Overall: max_abs=1.067e+19, max_rel=2.512e+00
  1. case_05714_seed65500185
Case: case_05714_seed65500185
Mismatched outputs (optimized vs reference):
	v10_0 (output[4]): max_abs=1.153e+00, max_rel=1.005e+00, shape=(1, 54)
Overall: max_abs=1.153e+00, max_rel=1.005e+00

Attachments

  • README.md (this document)
  • requirements.txt (dependency versions)
  • optimize_model.py (runs only eliminate_slice_after_shape and saves model.opt.onnx)
  • diff_test.py (runs original vs optimized with oracle inputs and reports max_abs/max_rel)
  • run_optimized_model.py (runs the optimized model to inspect outputs)
  • case_*/ (original models plus oracle inputs for reproduction)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions