Execution error of Convolution2d on EthosNAcc mode

[Rahn80643]

Hi,
We have an int8 tflite model based on BiSeNetv2 architecture. When we applied the model on NPU, we met the following error:
WARNING: Layer of type Convolution2d is not supported on requested backend EthosNAcc for input data type QAsymmS8 and output data type QAsymmS8(reason: Convolution: Overall scale (of the input* weights/ output) should be in range [2.328306e-10, 1)), falling back to the next backend.
Warning: ERROR: Layer of type Convolution2d is not supported on any preferred backend[EthosNAcc]
terminate called after throwing an instance of 'armnn::InvalidArgumentException'
what(): Failed to assign a backend to each layer

However, the same model could be executed in GPUAcc mode and SPA tool (with NPU evaluation). Does the error above mean the weight value of the convolution2d layer must in the range [2.328306e-10, 1)?

Best Regards,
Rahn

[mohelz01]

Hi,

reason: Convolution: Overall scale (of the input* weights/ output) should be in range [2.328306e-10, 1)
Please note that the quantization scale of input, weights, and output need to satisfy the above equation

Best Regards,
Mohamed

[Rahn80643]

Hi,
Since the int8 tflite model is converted based on (TensorFlow2 int8 calibration)[https://www.tensorflow.org/lite/performance/post_training_quantization#integer_only], and the model could successfully executed in both CpuAcc and GpuAcc backend. Is there any other step needed to be included to allow the weight values in the converted int8 tflite model satisfy the requirement of NPU?

Best Regard,
Rahn

[mohelz01]

Hi,
You mentioned your model is based on BiSeNetv2 architecture. I would like to produce the error on my environment. It would be helpful if you let me know your reproduce steps, the environment used, and if there's any model I can use that produce the same error.

Looking forward to hear from you

Best Regards,
Mohamed

[Rahn80643]

Hi,
The attachment is the BiSeNetv2 model we used in our experiment, and the following is the information about our environment.
AMR_20211018_office1_2_3_exp7Ext1_nosoftmax_int8.tflite.tar.gz

1. The model is tested on a ZEBU simulator with a cortex A55 CPU and a mounted ethos N78 device
2. The version of ArmNN, Compute Library, and ethos-n driver is 21.08.
3. Since our model contains a argmax layer which is not supported in NPU, we apply the "hint" mechanism to execute the layer on CpuACC mode, and the other layers executed on EthosACC mode.

Best Regard,
Rahn

[mohelz01]

Hi,

Please note that this is currently not supported. We are evaluating adding support for this in a future release. My colleague is looking to provide more details on the corresponding ticket opened in the support channel.

Best Regards,
Mohamed

[zingo]

Hi @Rahn80643 thanks for checking and helping out with our code!
I just want to check if you are you happy with this info for now or do you have anything you want to follow up with?
(I'm wondering if we can close this ticket for now?)

Best Regards,
Zingo

[Rahn80643]

Hi, @zingo
Thank your for your support. We don't have anything to follow up with currently, please close this ticket.

Best Regards,
Rahn