mcunet model with cmsis-nn

[brainchip-india]

Hi @meenchen @RaymondWang0 I couldn't find any tutorial or demo to run mcunet models with cmsis-nn, can you please point me to the page or please let me know how to generate mcunet model to deploy on cmsis-nn functions.. or can i use the same model with cmsis-nn similar functions?

[RaymondWang0]

Hi @brainchip-india,

The tutorial or demo to run models with CMSIS-NN is beyond the scope of this repo since this repo is mainly for TinyEngine. Our suggestion is: First, obtain the model you would like to use (for the MCUNet models, please refer to the model zoo in MCUNet repo). Then, please refer to TF-Lite Micro's repo and CMSIS-NN's repo to learn how to deploy models by using TF-Lite Micro's runtime and CMSIS-NN's optimized kernels.

[brainchip-india]

Hi @RaymondWang0 @meenchen , there are few observations when I tested with tflite @713b6ed6 when I create a static tflite library for the M7 board the inference is failing at allocate_tensors() function in tflite execution, however when I tried creating tflite static library for M4 board and ran the lib on M7 the build/inference got passed (please refer to the attachment) and almost matched to the measured results

my question: is it the expected procedure? or am I missing something here to run the tflite model on M7, please help me thanks in advance.

Edited: one more question:, the timing results in measured results are only inference time? or camera capture + conversion to 8bits + inference?

[meenchen]

Hi @brainchip-india,

We are using the source code of tflite micro during the measurement instead of the static library, but the measurement results you got seem reasonable.

Regarding another question, yes, the timing results are only for inference.

[brainchip-india]

Hi @meenchen, I have fixed the tflite M7 static library build now I am able to run the inference.

One last question:

With the tflite build(default kernal and cmsis-nn enabled) the inference results are not consistent compared to mcunet original model, however when running the tflite_eval.py it showing accuracy of 88% but the end inference results are not accurate/not consistent with live images any reason? same with static images I tried with 10 static person images with tflite the inference results are very poor(and I tried subtracting 128 also as per tflite_eval.py file/ live image conversion still the inference is poor).

[meenchen]

Hi @brainchip-india,

Unfortunately, I don't have a thorough understanding of the tflite micro. Based on what you have described, it seems that there may be an issue with the tflite micro. I suggest that you file an issue report on their GitHub repository in order to bring this to the attention of the developers and potentially get some assistance with resolving the problem.

[brainchip-india]

Hi @meenchen, I have resolved the issue actually I am doing an input scaling to the inputs (model_input->params.scale + model_input->params.zero_point) which is not required as per your tflite model and just subtracted -128 to the input which got solved the problem now I am able to see the accuracy is good with both live and static images.

one question: I don't see any time capture code in the project repo, are you using systick or any other external timer to calculate the inference time??

[brainchip-india]

Hi @meenchen, can you please share the new board details (STM32H743) I see there are lot of varients. it will be helpful if you can share the link.
Thanks.

[meenchen]

Hi @brainchip-india, we are using CoreH743: https://www.waveshare.com/coreh743i.htm.

[brainchip-india]

Hi @meenchen, thanks for the link and I see that your model zoo is not updated with the latest models assuming new build project is releasing soon with the new board and new models.
Any particular date?

[brainchip-india]

Hi @meenchen, with the latest PR merge of import path, I am still seeing the model generate failure can you please look into it.

Error:
Traceback (most recent call last):
File "examples/vww.py", line 20, in
from mcunet.mcunet.model_zoo import download_tflite
ModuleNotFoundError: No module named 'mcunet.mcunet'

[brainchip-india]

@meenchen
copying init.py to mcunet parent folder solved the problem able to download the model
command: checkout to tinyengine folder
cp mcunet/mcunet/init.py mcunet/

[meenchen]

Hi @brainchip-india,

Thanks for letting us know about the issue of accessing the mcunet submodule. However, I was not able to reproduce this error on my side. Are you following the steps from our tutorial? could you share some info on your dev environment (e.g., OS and python version)?

[brainchip-india]

I am using UBUNTU 20.0.4 and python as recommended in the tutorial 3.6+

[brainchip-india]

Hi @meenchen, @RaymondWang0 I tried the mcunet-vww-0 static image inference on the new board (STM32H743IIT6) and I see that the inference time result is 202ms which is almost 8 times more than the measured results of yours. Am I missing anything here? please let me know.

[meenchen]

Hi @meenchen, @RaymondWang0 I tried the mcunet-vww-0 static image inference on the new board (STM32H743IIT6) and I see that the inference time result is 202ms which is almost 8 times more than the measured results of yours. Am I missing anything here? please let me know.

Hi @brainchip-india, please double-check the following settings:

1. Make sure to use -O3 or -Ofast for the optimization level of C/C++.
2. Make sure you enable the D cache and I cache on the board.
3. Make sure your board is running at 480MHz.