QAT LSTM with hidden state (h0, c0) will get error

[juyenshun]

Hi,

I want to apply hidden state on my LSTM model and run QAT. The model can run normally without Quantizer, but will get the error below when I apply QATQuantizer:

ERROR (tinynn.graph.tracer) inputs: ['input_1', 'input_2', 'input_3']
ERROR (tinynn.graph.tracer) forwards: []
ERROR (tinynn.graph.tracer) outputs: []
ERROR (tinynn.graph.tracer) constants: []

The attachment is an example file, thanks.
torch_lstm_test.zip

[peterjc123]

Please refer to the FAQ for exporting LSTM to TFLite. https://github.com/alibaba/TinyNeuralNetwork/blob/main/docs/FAQ.md#how-to-convert-a-model-with-lstm
As a requirement, you must remove the state inputs and outputs from your model.

[peterjc123]

FYI, we only support dynamic quantization for the LSTM op. So, with your code, LSTM will still be evaluated with floating type numeric. If you want to mix QAT and dynamic quantization, then you may take a look at #81.

As for the model given in your example, dynamic quantization should be preferred. Please take a look at https://github.com/alibaba/TinyNeuralNetwork/blob/main/examples/quantization/dynamic.py .

[juyenshun]

Thanks for your information, I will try it!

[peterjc123]

This issue will be closed due to inactivity. If you still have problems, please feel free to reopen it. Thanks for your support.