print_qstats(): operation type issue with Sequential() model

[HaoranREN]

When I was applying quantization on a Keras Sequential() model, I found that there could be an issue about the operation type in print_stats() function.

For example, with the model in example_mnist.py but coded by the Sequential() API, I got an output as below. The operation type for the first conv2d layer is unull_4_-1, whereas it is smult_4_8 with the functional API.

Based on my experiments with some other models, this only happens to the first layer of the Sequential() model.

Also, for smult_4_8, I would like to know what does the 8 stand for here?

I am on:
tensorflow-gpu 2.2.0
tensorflow-model-optimization 0.4.1

Number of operations in model:
    conv2d_0_m                    : 25088 (unull_4_-1)
    conv2d_1_m                    : 663552 (smult_4_4)
    conv2d_2_m                    : 147456 (smult_4_4)
    dense                         : 5760  (smult_4_4)

Number of operation types in model:
    smult_4_4                     : 816768
    unull_4_-1                    : 25088

Weight profiling:
    conv2d_0_m_weights             : 128   (4-bit unit)
    conv2d_0_m_bias                : 32    (4-bit unit)
    conv2d_1_m_weights             : 18432 (4-bit unit)
    conv2d_1_m_bias                : 64    (4-bit unit)
    conv2d_2_m_weights             : 16384 (4-bit unit)
    conv2d_2_m_bias                : 64    (4-bit unit)
    dense_weights                  : 5760  (4-bit unit)
    dense_bias                     : 10    (4-bit unit)

Weight sparsity:
... quantizing model
    conv2d_0_m                     : 0.1812
    conv2d_1_m                     : 0.1345
    conv2d_2_m                     : 0.1156
    dense                          : 0.1393
    ----------------------------------------
    Total Sparsity                 : 0.1278

[zhuangh]

@danielemoro please take a look. thanks

[lishanok]

Hi @HaoranREN,
Thank you for your input.
smult_4_8 means s:signed multi:multiplication op, 4:weight quantizer bit, 8:input quantizer bit
Regarding the undesired operation type for a Sequential model, we are fixing it right now.

[HaoranREN]

@lishanok Thanks a lot for your explanations 👍

[danielemoro]

@HaoranREN I've been looking into your issue and created a fix. You should hopefully have access to it soon.

The reason why you see unull_4_-1 for the first layer is because QKeras cannot find the input tensor to the first layer. This means QKeras doesn't know the the bit size used by the input tensor, and it therefore cannot determine what hardware operation would be needed to process that input. That's why the mode is null, and the bits of the inputs tensor is -1.

The reason why this is happening in the Sequential API and not the Functional API is because the Input Layer in the Sequential API is "hidden" when iterating model layers, and so the tool does not cache the input bits that would be passed in to the first convolution layer. This is just a quirk of Keras, so the fix allows code needs to handle both situations.

Thank you for bringing this to our attention,

Daniele

[HaoranREN]

@danielemoro Thanks a lot for your response and it makes a lot of sense. I was also suspecting the Keras implementation, since like a said, it only happens to the first layer of the Sequential() model.

[zhuangh]

a fix was merged. close it. feel free to reopen it if you have any questions.