Skip to content

Adding ability to record intermediate graph outputs#1588

Merged
askhade merged 19 commits into
microsoft:askhade/quantization_and_caliberationfrom
agnikumar:agnikumar/calibration
Aug 20, 2019
Merged

Adding ability to record intermediate graph outputs#1588
askhade merged 19 commits into
microsoft:askhade/quantization_and_caliberationfrom
agnikumar:agnikumar/calibration

Conversation

@agnikumar
Copy link
Copy Markdown

@agnikumar agnikumar commented Aug 8, 2019

Calibration can be used to improve quantization techniques, adding reduced-precision computation for neural networks while retaining high accuracy without retraining. calibrate.py adds ReduceMin and ReduceMax nodes to all Conv and MatMul nodes in a loaded ONNX model and ensures their outputs are stored as part of the graph output, extracts intermediate output values after inference, and returns a dictionary mapping added node names to average (ReduceMin, ReduceMax) values as input to quantize.py. Example usage:

python calibrate.py --model_path=<'path/to/model.onnx'> --data_set_path=<'path/to/data/folder'> --calib_mode='naive'

@agnikumar agnikumar requested a review from a team as a code owner August 8, 2019 15:02
@msftclas
Copy link
Copy Markdown

msftclas commented Aug 8, 2019

CLA assistant check
All CLA requirements met.

Comment thread onnxruntime/python/tools/quantization/calibrate.py Outdated
Comment thread onnxruntime/python/tools/quantization/calibrate.py Outdated
Comment thread onnxruntime/python/tools/quantization/calibrate.py Outdated
Comment thread onnxruntime/python/tools/quantization/calibrate.py
Comment thread onnxruntime/python/tools/quantization/calibrate.py Outdated
@jywu-msft
Copy link
Copy Markdown
Member

add some tests?

Comment thread onnxruntime/python/tools/quantization/calibrate.py Outdated
Comment thread onnxruntime/python/tools/quantization/README.md Outdated
Comment thread onnxruntime/python/tools/quantization/README.md Outdated
Comment thread onnxruntime/python/tools/quantization/calibrate.py Outdated
Comment thread onnxruntime/python/tools/quantization/calibrate.py Outdated
Comment thread onnxruntime/python/tools/quantization/calibrate.py Outdated
Comment thread onnxruntime/python/tools/quantization/calibrate.py Outdated
Comment thread onnxruntime/python/tools/quantization/calibrate.py
Comment thread onnxruntime/python/tools/quantization/smooth_average.py Outdated
Comment thread onnxruntime/python/tools/quantization/calibrate.py Outdated
Comment thread onnxruntime/python/tools/quantization/calibrate.py Outdated
Comment thread onnxruntime/python/tools/quantization/calibrate.py
Comment thread onnxruntime/python/tools/quantization/calibrate.py
Comment thread onnxruntime/python/tools/quantization/calibrate.py Outdated
Comment thread onnxruntime/python/tools/quantization/calibrate.py Outdated
Comment thread onnxruntime/python/tools/quantization/calibrate.py Outdated
Comment thread onnxruntime/python/tools/quantization/calibrate.py Outdated
@askhade
Copy link
Copy Markdown
Contributor

askhade commented Aug 19, 2019

In our last discussion we decided not to replace the existing code. Looks like your PR is replacing the whole existing script. Can you please make that change? Everything else looks good but I cant merge this till you make this change.... Thanks!

@askhade askhade merged commit 963caac into microsoft:askhade/quantization_and_caliberation Aug 20, 2019
@askhade askhade mentioned this pull request Jan 16, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants