Run a YOLOv8 model on a Himax Microcontroller

content/learning-paths/microcontrollers/yolo-on-himax/_index.md

@@ -0,0 +1,40 @@
+---
+title: Run a Computer Vision Model on a Himax Microcontroller
+
+minutes_to_complete: 90
+
+who_is_this_for: This is an introduction topic for beginners on how to run a computervision application on an embedded device from Himax. This example uses an off-the-shelf Himax WiseEye2 module which is based on the Arm Cortex-M55 and Ethos-U55.
+
+learning_objectives: 
+    - Run a you-only-look-once (YOLO) computer vision model using off-the-shelf hardware based on the Arm Cortex-M55 and Ethos-U55.
+    - Learn how to build the Himax SDK and generate firmware image file.
+    - Learn how to update firmware on edge device (Himax WiseEye2).
+
+prerequisites:
+    - Seeed Grove Vision AI V2 Module
+    - OV5647-62 Camera module and included FPC cable
+    - A USB-C cable
+    - A Linux/Windows-based PC on an x86 archiecture. 
+
+author_primary: Chaodong Gong, Alex Su, Kieran Hejmadi
+
+### Tags
+skilllevels: Beginner
+subjects: ML
+armips:
+    - Cortex M55
+    - Ethos U55
+tools_software_languages:
+    - Himax SDK
+    - Bash
+operatingsystems:
+    - Linux
+    - Windows
+
+
+### FIXED, DO NOT MODIFY
+# ================================================================================
+weight: 1                       # _index.md always has weight of 1 to order correctly
+layout: "learningpathall"       # All files under learning paths have this same wrapper
+learning_path_main_page: "yes"  # This should be surfaced when looking for related content. Only set for _index.md of learning path content.
+---

content/learning-paths/microcontrollers/yolo-on-himax/_next-steps.md

@@ -0,0 +1,22 @@
+---
+next_step_guidance: Navigate Machine Learning Development with Ethos-U processors
+
+recommended_path: /learning-paths/microcontrollers/nav-mlek/
+
+further_reading:
+    - resource:
+        title: Grove Vision AI Module V2 User Documentation
+        link: https://wiki.seeedstudio.com/grove_vision_ai_v2/
+        type: documentation
+    - resource:
+        title: WiseEye2 HX6538 processor blog (SoC powering Grove Vision AI Module V2) 
+        link: https://www.himax.com.tw/products/wiseeye-ai-sensing/wiseeye2-ai-processor/
+        type: blog
+
+# ================================================================================
+#       FIXED, DO NOT MODIFY
+# ================================================================================
+weight: 21                  # set to always be larger than the content in this path, and one more than 'review'
+title: "Next Steps"         # Always the same
+layout: "learningpathall"   # All files under learning paths have this same wrapper
+---

content/learning-paths/microcontrollers/yolo-on-himax/_review.md

@@ -0,0 +1,20 @@
+---
+review:
+    - questions:
+        question: >
+            The Grove Vision AI V2 Module can run Yolov8 model in real time?
+        answers:
+            - True
+            - False
+        correct_answer: 1                    
+        explanation: >
+            The Grove Vision AI V2 Module can run object detection in real time using the Cortex-M55 and Ethos-U55. 
+
+
+# ================================================================================
+#       FIXED, DO NOT MODIFY
+# ================================================================================
+title: "Review"                 # Always the same title
+weight: 20                      # Set to always be larger than the content in this path
+layout: "learningpathall"       # All files under learning paths have this same wrapper
+---

content/learning-paths/microcontrollers/yolo-on-himax/how-to-1.md

@@ -0,0 +1,88 @@
+---
+title: Set Up Environment
+weight: 2
+
+### FIXED, DO NOT MODIFY
+layout: learningpathall
+---
+
+## Set up the Development Environment
+
+### Step 1.1. Install Ubuntu
+
+If you are running Windows on your host machine, we recommend using Ubuntu through Windows subsystem for Linux 2 (WSL2). Please see [this learning path](https://learn.arm.com/learning-paths/laptops-and-desktops/wsl2/setup/) for assistance  
+
+This learning path has been validated on Ubuntu 22.04 LTS. However, we expect other linux distributions to work. To verify the Linux distribution you are using you can run the `cat /etc/*release*` command. 
+
+```bash
+cat /etc/*release*
+```
+The top lines from the terminal output will show the distribution version. 
+
+```output
+DISTRIB_ID=Ubuntu
+DISTRIB_RELEASE=22.04
+DISTRIB_CODENAME=jammy
+DISTRIB_DESCRIPTION="Ubuntu 22.04.5 LTS"
+...
+```
+
+### Step 1.2. (Optional) Install Microsoft Visual Studio Code
+
+This is only optional. You can use any text editor you are comfortable with to view or edit code. By typing “wsl” in VS Code terminal, you can switch to Linux environment.
+
+### Step 1.3. Install python 3
+
+Go to website python.org to download and install.
+Verify python is installed by
+python3 --version
+You should see an output like the following.
+```output
+Python 3.12.7
+```
+### Step 1.4. Install python-pip
+
+```bash
+sudo apt update
+sudo apt install python3-pip -y
+pip3 --version
+```
+
+If `pip3` is correctly installed you should see an output similar to tht following.
+
+```output
+pip 24.2 from <path to pip3>/pip (python 3.12)
+```
+
+### Step 1.5. Install make
+
+You will need to install the make build tool in order to build the firmware in the following section.
+
+```bash
+sudo apt update
+sudo apt install make -y
+```
+
+Successful installation of make will show the following when the `make --version` command is run.
+
+```output
+$ make --version
+GNU Make 4.3
+Built for x86_64-pc-linux-gnu
+Copyright (C) 1988-2020 Free Software Foundation, Inc.
+License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
+This is free software: you are free to change and redistribute it.
+There is NO WARRANTY, to the extent permitted by law.
+```
+
+### Step 1.6. Install ARM GNU toolchain
+
+```bash
+cd ~
+wget https://developer.arm.com/-/media/Files/downloads/gnu/13.2.rel1/binrel/arm-gnu-toolchain-13.2.rel1-x86_64-arm-none-eabi.tar.xz
+tar -xvf arm-gnu-toolchain-13.2.rel1-x86_64-arm-none-eabi.tar.xz
+export PATH="$HOME/arm-gnu-toolchain-13.2.Rel1-x86_64-arm-none-eabi/bin/:$PATH"
+```
+
+Please note: you may want to add the command to your `bashrc` file. This enables the Arm GNU toolchain to be easily accessed from any new terminal session. 
+

content/learning-paths/microcontrollers/yolo-on-himax/how-to-2.md

@@ -0,0 +1,60 @@
+---
+title: Build The Firmware
+weight: 3
+
+### FIXED, DO NOT MODIFY
+layout: learningpathall
+---
+
+## Build The Firmware
+
+Next, we need to build an image that contains the embedded software (firmware). You will need to have the git version control system installed. Run the command below to verify that git is installed on your system.
+
+```bash
+git --version
+```
+
+You should see output similar to that below.
+
+```output
+git version 2.39.3
+```
+
+If not, please follow the steps to install git on your system.
+
+### Step 2.1. Clone the Himax project
+
+You will first need to recusively clone the Himax repository. This will also clone the necessary sub repos such as Arm CMSIS. 
+
+```bash
+git clone --recursive https://github.com/HimaxWiseEyePlus/Seeed_Grove_Vision_AI_Module_V2.git
+cd Seeed_Grove_Vision_AI_Module_V2
+```
+
+### Step 2.2. Compile the Firmware
+
+The make build tool is used to compile the source code. This should take up around 2-3 minutes depending on the number of CPU cores available.
+
+```bash
+cd EPII_CM55M_APP_S
+make clean
+make
+```
+
+
+### Step 2.3. Generate a Firmware Image
+
+```bash
+cd ../we2_image_gen_local/
+cp ../EPII_CM55M_APP_S/obj_epii_evb_icv30_bdv10/gnu_epii_evb_WLCSP65/EPII_CM55M_gnu_epii_evb_WLCSP65_s.elf input_case1_secboot/
+./we2_local_image_gen project_case1_blp_wlcsp.json
+```
+
+Your terminal output should end with the following.
+
+```output
+Output image: output_case1_sec_wlcsp/output.img
+Output image: output_case1_sec_wlcsp/output.img
+
+IMAGE GEN DONE
+```

content/learning-paths/microcontrollers/yolo-on-himax/how-to-3.md

@@ -0,0 +1,46 @@
+---
+title: Flash Firmware onto the Microcontroller
+weight: 3
+
+### FIXED, DO NOT MODIFY
+layout: learningpathall
+---
+
+## Flash the Firmware
+
+Now that we have generated a firmware file on our local machine, we need to flash the microcontroller with this firmware. 
+
+### Step 3.1. Install xmodem.
+
+`Xmodem` is a basic file transfer protocol. Run the following command to install the dependencies for xmodem. 
+
+```bash
+cd $HOME/Seeed_Grove_Vision_AI_Module_V2 # If you cloned the repo to a different location replace $HOME with the path. 
+pip install -r xmodem/requirements.txt
+```
+
+### Step 3.2. Connect the module to PC by USB cable.
+
+You will need to insert the FPC cable cable into the Grove Vision AI V2 module. Lift the dark grey latch on the connector as per the image below.
+
+![unlatched](./unlatched.jpg)
+
+Then, slide the FPC connector in with the metal pins facing down and close the dark grey latch to fasten the connector. 
+
+![latched](./latched.jpg)
+
+Then connect the Groove Vision AI V2 Module to your computer via the USB-C cable.
+
+### Step 3.4. Flash the firmware onto the moule.
+
+Run the python script below to flash the firmware. 
+
+```python
+python xmodem\xmodem_send.py --port=[your COM number] --baudrate=921600 --protocol=xmodem --file=we2_image_gen_local\output_case1_sec_wlcsp\output.img 
+```
+
+    Note: If running one of the other example models demonstrated in '(Optional) Try Different Models', the command might be slightly different.
+
+After the firmware image burning is completed, the message "Do you want to end file transmission and reboot system? (y)" is displayed. Press the reset button on the module as per the image below.
+
+![reset button](./reset_button.jpg)

content/learning-paths/microcontrollers/yolo-on-himax/how-to-4.md

@@ -0,0 +1,26 @@
+---
+title: Run and View Model Results
+weight: 3
+
+### FIXED, DO NOT MODIFY
+layout: learningpathall
+---
+
+
+### Step 4.1. Connect module to PC with USB cable.
+
+Exit the terminal session and connect the module to the PC via your USB-C cable. 
+
+### Step 4.2. Download the Himax AI web toolkit.  
+
+The Himax AI web toolkit enables a browser-based graphical user interface (GUI) for the live camera feed. 
+
+Download the Himax AI Web toolkit by clicking on this [link](https://github.com/HimaxWiseEyePlus/Seeed_Grove_Vision_AI_Module_V2/releases/download/v1.1/Himax_AI_web_toolkit.zip)
+
+Unzip the archived file and double click `index.html`. This will open the GUI within your default browser.
+
+### Step 4.3. Connect to the Grove Vision AI
+
+Select 'Grove Vision AI(V2)' in the top-right hand corner and press connect button. 
+
+![Himax web UI](./himax_web_ui.jpg)

content/learning-paths/microcontrollers/yolo-on-himax/how-to-5.md

@@ -0,0 +1,41 @@
+---
+title: (Optional) Try Different Models
+weight: 5
+
+### FIXED, DO NOT MODIFY
+layout: learningpathall
+---
+
+
+### Modify the makefile
+
+Change the directory to the where the makefile is located.
+
+```bash
+cd $HOME/Seeed_Grove_Vision_AI_Module_V2/EPII_CM55M_APP_S/ # replace $HOME with the location of the project
+```
+
+Using a text editor, for example visual studio code or nano, modify the `APP_TYPE` field in the makefile from the default value of `allon_sensor_tflm` to one of the values in the table below 
+
+
+|APP_TYPE =|Description|
+|---|---|
+|tflm_folov8_od|Object detection|
+|tflm_folov8_pose|Pose detection|
+|tflm_fd_fm|Face detection|
+
+### Regenerate the Firmware Image
+
+Go back to the 'Build The Firmware' section and start from Step 3.2. to regenerate the firmware image. 
+
+The images below are examples images from the model. 
+
+#### Objection Detection
+![object_detection](./object_detection.jpg)
+
+#### Pose Estimation
+![Pose estimation](./pose_estimation.jpg)
+
+#### Face Detection
+![object_detection](./face_detection.jpg)
+