what exactly are htp_quantsim_config_v#.json?

[JerryLee (jl749)]

Hello AIMET team

When I am working with QuantsimModel I see that we are passing config_file
If I understand correctly config jsons under aimet_common specifies QDQ placements/schemes for each Hexagon Arch.

When I export v75 compatible QNN encoding + ONNX file following llama3.2-3b deployment example under qaihub-model

# e.g.
python3 -m qai_hub_models.models.llama_v3_2_3b_chat_quantized.export \
  --skip-inferencing \
  --skip-profiling \
  --output-dir "genie_bundle" \
  --device "Samsung Galaxy S24 Ultra"

It produces an encoding file containing

• per-channel-quantized MatMul
  
• output-quantized Gather
  

Which contradicts htp_quantsim_config_v75.json
because

• per_channel_quantization is disabled for MatMul in v75 config
  https://github.com/quic/aimet/blob/22d47e49556168dbebdd29f3a4325ddbc512701e/TrainingExtensions/common/src/python/aimet_common/quantsim_config/htp_quantsim_config_v75.json#L129-L132
• output_quantization is disabled for Gather in v75 config
  https://github.com/quic/aimet/blob/22d47e49556168dbebdd29f3a4325ddbc512701e/TrainingExtensions/common/src/python/aimet_common/quantsim_config/htp_quantsim_config_v75.json#L81-L85
• I am assuming qaihub-model's pre-computed encodings were generated by AIMET.

Is my understanding of config.json wrong?
Any guidance you can provide me would be appreciated.

[JerryLee (jl749)]

I have two additional questions which are related to my question above so I am posting them here as well:

1. The diagram below was generated by QNN context binary visualizer
   is the output of the gather node quantized to quint16 because the child node's input_quantizer has been enabled?
   

2. I do not see any block quantization related restrictions under the config files. Where can I find QNN block quantization requirements? Namely:
   what are the supported block_sizes (32, 64, 128 ?)
   what hexagon devices support block quant? (is block quant supported on v68?)