the usage of the .encodings file produced by AIMET

[wwwsssaadd (xs-alt)]

Hi all,
l have some questions when using the AMP algorithm for onnx and convert the model to QNN by using qnn-onnx-converter

1. the AMP algorithm will produce an encodings file and it contains bitwidth/dtype/scale/offset and other properties for quantization usage, does it contains all nodes' quantization info or it gives some nodes' quantization scheme and leaves others to be calculated during the normal quantization process
2. When using --quantization_overrides with qnn-onnx-converter tool, how to set the --bias_bitwidth, --act_bitwidth, --weights_bitwidth and other related params so that it can produce the same results like the quantsim model in AIMET, it is related to the param_default_bitwidth and act_default_bitwidth when creating QuantizationSimModel?
3. Is it possible to constrain a specific node or certain types of nodes' quant bitwith and dtype when doing AMP and how.

[Michael Tuttle (quic-mtuttle)]

Hi wwwsssaadd (@xs-alt), good questions.

1. The AMP encodings file should be complete; no additional quantization is necessary.

2. You should almost always set --bias_bitwidth as 32. --act_bitwidth corresponds to default_activation_bw and --weights_bitwidth corresponds to default_param_bw, but these two will only come into play if there are some encodings missing from the quantization overrides that you want QNN to fill in. Otherwise, you can pass the --float_fallback flag to run layers with missing encodings in floating-point.

3. This isn't well supported at the moment, but it is something we plan to add.

[wwwsssaadd (xs-alt)]

Michael Tuttle (@quic-mtuttle) Thanks for the answers!
l have one more question. Is it possible to set a quantsim model's certain node in float mode and export it to the encodings file?

[Michael Tuttle (quic-mtuttle)]

Yes, this is technically possible, but its not really an advertised/public API since doing this can cause the qnn converter to fail with the exported encodings in some cases. If you would like to try it at your own risk anyway, it would look something like

from aimet_onnx.quantsim import QuantizationDataType

quantsim: QuantizationSimModel = ... # Your quantsim model
tensor_name: str = ... # name of some node input or output in the onnx graph

if tensor_name in quantsim.qc_quantize_op_dict:
     sim.qc_quantize_op_dict[tensor_name].data_type = QuantizationDataType.float
     sim.qc_quantize_op_dict[tensor_name].set_bitwidth(16)