Esspressif - ESP - IDF

[christophezeinaty]

Hello everyone, I am new to larq and esspressif and I want to ask if anyone have any recommendation on how to integrate the compute-engine in my esspressif project I can't find any documentation on that specifically.

Thank you in advance

[CNugteren]

Hello christophezeinaty. The Larq compute engine is meant more for mobile-class devices with CPUs such as the Arm Cortex-A. My guess is that your Espressif platform uses a microcontroller of the ESP32 family, which isn't the target for the Larq compute engine, so unfortunately we can't help you there.

If you are interested in running models on your Espressif platform, Plumerai does offer commercial solutions. For more information, see our docs on the inference engine or contact us directly.

[christophezeinaty]

Thank you for your response, that's true I saw that in the documentation, but if I understood correctly lets say I am able to extract the binary weights and activations I can always do the inference using bitwise operations but I wont be able to use directly compute-engine? what I mean is that I can always use larq as platform for my Binarized neural network is that correct?

[CNugteren]

I'm not sure what you mean exactly. So you want to train a model with BNN layers using Larq, and then run inference without the Larq compute engine? No, that is not possible, unless you implement your own custom-ops in TFLite or TFLM.

[christophezeinaty]

lets say I train a model using larq that have conv2d layers, I save these weights and reload them in another model that have the same architecture but not using the larq conv2d, will it be possible to remove matrix multiplications since I have the binary weights and use the tensorflow lite ops for conv2d for example?

[CNugteren]

So you would train a BNN layer with BConv2D layers, then write some script to convert those BConv2D layers to regular Conv2D with all weights being binary but represented as int8 or float32, and then you would run with a normal inference engine such as TFLM? Yes that seems possible in theory, although it does seem like quite some work. But then you also want to 'remove matrix multiplications', but that would mean that you modify the code of your inference engine (e.g. TFLM) to do something special in those cases? Sounds like a lot of work. In that case, it might just be easier to port the custom BConv2D ops from the Larq compute engine to TFLM or so, but that is also a lot of work.

[christophezeinaty]

Yes I understand, for the moment I was able to train a BNN on a machine based on x86 but the problem with that is that I can't evaluate the inference results using larq compute-engine to be able to compare the inference result with a quantized int8 network, so I need to find a way to emulate the performance on x86 to see the real benefit from the BNN and ideally I want to deploy it on the esp32.

Correct lot of work to be done .... but for the moment I didn't find easier way.

PS: I am trying the online inference engine so at least I have some results based on arm architecture its really interesting.

Thank you for your help