About this respository

This project aims to deploy a TensorFlow lite model into TTGO T-Camera ESP32 WROVER using Micropython. The given approach use Thonny as a Python IDE to install the firmware and compile the scripts.
This project was based on @moclieri https://github.com/mocleiri/tensorflow-micropython-examples/tree/main/examples/person_detection and thanks to @uraich https://github.com/uraich/EdgeComputing/tree/main/firmware for fixing the camera module issues presented in the recent MicroPython firmware. For more information please visit their respository.

Uploading firmware into ESP32

Any Python compiler should work with this project.
However, this guide will use the latest Thonny IDE version as an intuition for new users, visit https://thonny.org/.

Click tools and go to option, set the interpreter to 'Micropython (ESP32)' and select your ESP port.
Next click on 'Install or update Micropython' on the bottom right of the window.

Upload the firmware bin. The file can be found inside the firmware folder with .bin extension

Thanks @uraich again for the recent fix with the given firmware.

Once the firmware is uploaded, press the boot button to reset ESP32 and test the firmware by importing the given library.
All imports should not raise any errors.

Compiling the scripts

Upload all files in 'Micropython-scripts' folder into ESP-32.

If 'device is busy' reset the ESP-32 by pressing the rst button, or interupt Thonny's backend by pressing ctr+c.

Run main.py script

If there is an 'Error when initializing the camera', simply reset your ESP32.

About the scripts

To write and update the SSD1306 OLED screen, a function with three input parameters was used.
The first input is a string variable which displays text into OLED screen.
Second input positions the texts in x-direction(horizontal) of the OLED screen.
Third input was used to position the texts in y-direction(vertical) of the OLED screen.

Thanks to Random Nerd Tutorials for providing the SSD1306 OLED display file, for more information please visit their site https://randomnerdtutorials.com/micropython-oled-display-esp32-esp8266/

Since I'm using TTGO T-Camera ESP32 WROVER, the camera pins was assign as followings. Please check your camera pins accordingly.

If an 'Error when initializing the camera' still persisted after resetting esp32, please check your camera pins.

Input_callback function setup bytes for an input Tensor. In other words it prepares the input picture captured by the camera to be used with AI model.

Output_callback function retrives AI condfidence values and print them to OLED screen. Microlite Outputs a Tensor value in signed 8-bit integer, namely from -128 to 127. We will convert this in a more readable manner, from 0 to 100 % instead.
If the confidence values is more than or equal to 52 %, change the color of the screen to white and display 'Person found!'
If no person is found or confidence is below 52 %, flip the screen's color the black and display 'No person found'

This sections primarily opens and prepares the Tensorflow-Lite model to be used in the program. The bytearray in line 58 depends on the size of the model.
microlite interpreter in line 63 has 4 input paramters.

Size of the input model, in this case we already prepares the model in previous section.
Memory space for inferencing the Tensorflow-Lite model, again this depends greatly on the size of the model.
Callback function for input tensor.
Callback fucntion for the output tensor.

In the main loop the program runs as followings:

1. Get machine time in seconds, use for calculating FPS.
   
2. Use the camera to capture an image.
   
3. Get ouput from the AI in accordance with the captured image.
   
4. Display FPS value on the bottom of the OLED screen.
   
5. Reset OLED screen so it can be updated with new texts/values.
   

Credits

Thanks to @moecleiri for providing the scripts and firmware for this respository https://github.com/mocleiri/tensorflow-micropython-examples/tree/main/examples/person_detection.
Thanks to @uraich https://github.com/uraich/EdgeComputing/tree/main/firmware for fixing the camera module issues presented in the recent MicroPython firmware.
Thanks to Random nerd tutorial again for providing SSD1306 OLED screen Micropython file. https://randomnerdtutorials.com/micropython-oled-display-esp32-esp8266/.