Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
Heterogeneous partitioning and quantization have issues when used together #3112
I observe that an attempt to run an NN model on heterogeneous devices using a quantization profile produced before on a single device (e.g. CPU) does not always work as expected because after applying the heterogeneous partitioning some nodes are not found in the saved quantization profile and thus cannot be quantized.
The solution we discussed looks something like this:
Later on, to run the quantized model, one would use the usual command-line with