[Tensorflow] Question: PTQ and QAT

[peiwenhuang27]

Hi, may I ask some questions based on my understanding of the source code please:

1. Conv2D

As far as I know, in post-training quantization, Conv2D supports both Conv2DBiasAddRelu and Conv2DBiasAddLeakyRelu through FuseNodeStartWithConv2d.apply_conv_biasadd_relu_fusion(...). However, the key difference is that with Leaky ReLU, quantized values cannot be directly passed to next quantized Conv2D due to the positive inputs constraint, so QuantizedConv2DWithBiasAndRelu will first dequantize to pass through Leaky ReLU, and then quantize again into next QuantizedConv2DWithBiasAndRelu.

So, if I have a quantization-aware trained model with Conv2DBiasAddLeakyRelu pattern, is it also converted to quantized model in the same manner? That is, regardless of the quantization method, in order to pass through Leaky ReLU, the predecessor node must first dequantize and the successor node must add a quantize input layer, is that correct?

2. LSTM

I noticed the following lines:

neural-compressor/neural_compressor/adaptor/tf_utils/quantize_graph/quantize_graph_matmul.py

Lines 121 to 125 in 1bddfcb

	# FIXME We only quantize the MatMul op which second input node type is const. This is a
	# workaround for RNN model like LTSM.
	if weight_node.op != 'Const':
	self.output_graph = self.input_graph
	return []

Does this mean quantization for LSTM is currently not supported?

Thanks!

[guomingz]

For Conv2D questions, i don't think there's need to insert additional dequantize/quantize before next QuantizedConv2DWithBiasAndRelu as this op supports s8 input.

For LTSM, i remember we already supported LTSM mode since v1.6 release

[ftian1]

close it if no further questions