diff --git a/content/learning-paths/microcontrollers/yolo-on-himax/_index.md b/content/learning-paths/microcontrollers/yolo-on-himax/_index.md index 1dcf71ef7a..f1c4abe72e 100644 --- a/content/learning-paths/microcontrollers/yolo-on-himax/_index.md +++ b/content/learning-paths/microcontrollers/yolo-on-himax/_index.md @@ -3,18 +3,18 @@ title: Run a Computer Vision Model on a Himax Microcontroller minutes_to_complete: 90 -who_is_this_for: This is an introduction topic for beginners on how to run a computer vision application on an embedded device from Himax. This example uses an off-the-shelf Himax WiseEye2 module which is based on the Arm Cortex-M55 and Ethos-U55. +who_is_this_for: This is an introduction topic explaining how to run a computer vision application on an embedded device from Himax. The example uses an off-the-shelf Himax WiseEye2 module which is based on Arm Cortex-M55 and Ethos-U55. learning_objectives: - - Run a you-only-look-once (YOLO) object detection model on the edge device - - Build the Himax Software Development Kit (SDK) and generate the firmware image file - - Update the firmware on the edge device (Himax WiseEye2) + - Run a you-only-look-once (YOLO) object detection model on the Himax device. + - Build the Himax Software Development Kit (SDK) and generate the firmware image file. + - Update the firmware on the Himax WiseEye2. prerequisites: - - A [Seeed Grove Vision AI Module V2](https://www.seeedstudio.com/Grove-Vision-AI-Module-V2-p-5851.html) development board - - A [OV5647-62 Camera Module](https://www.seeedstudio.com/OV5647-69-1-FOV-Camera-module-for-Raspberry-Pi-3B-4B-p-5484.html) and included FPC cable - - A USB-C cable - - An x86 based Linux machine or a machine running Apple Silicon + - A [Seeed Grove Vision AI Module V2](https://www.seeedstudio.com/Grove-Vision-AI-Module-V2-p-5851.html) development board. + - An [OV5647-62 Camera Module](https://www.seeedstudio.com/OV5647-69-1-FOV-Camera-module-for-Raspberry-Pi-3B-4B-p-5484.html) and included FPC cable. + - A USB-C cable. + - An x86 Linux machine or a Mac running macOS with Apple Silicon. author_primary: Chaodong Gong, Alex Su, Kieran Hejmadi diff --git a/content/learning-paths/microcontrollers/yolo-on-himax/build-firmware.md b/content/learning-paths/microcontrollers/yolo-on-himax/build-firmware.md index ef89acd6d3..b1db98e5f5 100644 --- a/content/learning-paths/microcontrollers/yolo-on-himax/build-firmware.md +++ b/content/learning-paths/microcontrollers/yolo-on-himax/build-firmware.md @@ -6,11 +6,15 @@ weight: 3 layout: learningpathall --- -This section will walk you though the process of generating the firmware image file. +This section explains the process of generating a firmware image file. -## Clone the Himax project +## Clone the Himax GitHub project -Himax has set up a repository containing a few examples for the Seeed Grove Vision AI V2 board. It contains third-party software and scripts to build and flash the image with the object detection application. By recursively cloning the Himax examples repo, git will include the necessary sub-repositories that have been configured for the project. +Himax maintains a repository containing a few examples for the Seeed Grove Vision AI V2 board. + +It contains third-party software and scripts to build and flash the image with the object detection application. By recursively cloning the Himax examples repo, git will include the necessary sub-repositories that have been configured for the project. + +Clone the repository: ```bash git clone --recursive https://github.com/HimaxWiseEyePlus/Seeed_Grove_Vision_AI_Module_V2.git @@ -19,41 +23,41 @@ cd Seeed_Grove_Vision_AI_Module_V2 ## Compile the firmware -For the object detection to activate, you need to edit the project's `makefile`, located in the `EPII_CM55M_APP_S` directory. +Use Make to compile the source code for object detection. -```bash -cd EPII_CM55M_APP_S -``` - -Use the `make` build tool to compile the source code. This should take up to 10 minutes depending on the number of CPU cores available on your host machine. The result is an `.elf` file written to the directory below. +This takes up to 10 minutes depending on the number of CPU cores available on your host machine. ```bash -make clean +cd EPII_CM55M_APP_S make ``` +When the build is complete, you have an `.elf` file at `obj_epii_evb_icv30_bdv10/gnu_epii_evb_WLCSP65/EPII_CM55M_gnu_epii_evb_WLCSP65_s.elf` + ## Generate the firmware image -The examples repository contains scripts to generate the image file. Copy the `.elf` file to the `input_case1_secboot` directory. +The examples repository contains scripts to generate the image file. + +Copy the `.elf` file to the `input_case1_secboot` directory. ```bash cd ../we2_image_gen_local/ cp ../EPII_CM55M_APP_S/obj_epii_evb_icv30_bdv10/gnu_epii_evb_WLCSP65/EPII_CM55M_gnu_epii_evb_WLCSP65_s.elf input_case1_secboot/ ``` -Run the script corresponding to the OS of your host machine. This will create a file named `output.img` in the `output_case1_sec_wlcsp` directory. +Run the script your OS as shown below. This will create a file named `output.img` in the `output_case1_sec_wlcsp` directory. {{< tabpane code=true >}} {{< tab header="Linux" language="shell">}} ./we2_local_image_gen project_case1_blp_wlcsp.json {{< /tab >}} - {{< tab header="MacOS" language="shell">}} + {{< tab header="macOS" language="shell">}} ./we2_local_image_gen_macOS_arm64 project_case1_blp_wlcsp.json {{< /tab >}} {{< /tabpane >}} -Your terminal output should end with the following. +The script output ends with the following output: ```output Output image: output_case1_sec_wlcsp/output.img @@ -62,4 +66,4 @@ Output image: output_case1_sec_wlcsp/output.img IMAGE GEN DONE ``` -With this step, you are ready to flash the image onto the Himax development board. \ No newline at end of file +You are ready to flash the image onto the Himax development board. \ No newline at end of file diff --git a/content/learning-paths/microcontrollers/yolo-on-himax/dev-env.md b/content/learning-paths/microcontrollers/yolo-on-himax/dev-env.md index 450297b752..73cb55e14f 100644 --- a/content/learning-paths/microcontrollers/yolo-on-himax/dev-env.md +++ b/content/learning-paths/microcontrollers/yolo-on-himax/dev-env.md @@ -1,48 +1,59 @@ --- -title: Set up environment +title: Set up the environment weight: 2 ### FIXED, DO NOT MODIFY layout: learningpathall --- -# Set up the development environment - -This learning path has been validated on Ubuntu 22.04 LTS and macOS. +This Learning Path has been validated on Ubuntu 22.04 LTS and macOS. {{% notice %}} -If you are running Windows on your host machine, you can use Ubuntu through Windows subsystem for Linux 2 (WSL2). Check out [this learning path](https://learn.arm.com/learning-paths/laptops-and-desktops/wsl2/setup/) to get started. +If you are running Windows, you can use Ubuntu through Windows subsystem for Linux 2 (WSL2). Check out [Get started with Windows Subsystem for Linux (WSL) on Arm](https://learn.arm.com/learning-paths/laptops-and-desktops/wsl2/setup/) to learn more. {{% /notice %}} -## Install Python, pip and git +## Install software tools + +Follow the instructions below to install the required development tools. + +### Install Python and Pip + +You will use Python to build the firmware image and pip to install additional dependencies. + +Verify Python is installed by running: -You will use Python to build the firmware image and pip to install some dependencies. Verify Python is installed by running ```bash python3 --version ``` -You should see an output like the following. +You should see an output like the following: + ```output Python 3.12.7 ``` -Install `pip` and `venv` with the following commands. +On Ubuntu, you may need to install `pip` and `venv` with the following commands: ```bash sudo apt update sudo apt install python3-pip python3-venv -y ``` -check the output to verify `pip` is installed correctly. -``` +Verify Pip is installed correctly: + +```bash pip3 --version ``` +The output is similar to: + ```output -pip 24.2 from //pip (python 3.12) +pip 24.0 from /usr/lib/python3/dist-packages/pip (python 3.12) ``` -It is considered good practice to manage `pip` packages through a virtual environment. Create one with the steps below. +It is good practice to manage Python packages through a virtual environment. + +Create one with the steps below. ```bash python3 -m venv $HOME/yolo-venv @@ -51,36 +62,43 @@ source $HOME/yolo-venv/bin/activate Your terminal displays `(yolo-venv)` in the prompt indicating the virtual environment is active. -You will need to have the git version control system installed. Run the command below to verify that git is installed on your system. +You also need the Git distributed version control system installed. + +Run the command below to verify that Git is installed on your system: ```bash git --version ``` -You should see output similar to that below. +If it is installed, you will see output similar to: ```output git version 2.39.3 ``` -## Install make +### Install Make -Install the make build tool, which is used to build the firmware in the next section. +Install the Make build tool, which is used to build the firmware in the next section. {{< tabpane code=true >}} {{< tab header="Linux" language="shell">}} sudo apt update sudo apt install make -y {{< /tab >}} - {{< tab header="MacOS" language="shell">}} + {{< tab header="macOS" language="shell">}} brew install make {{< /tab >}} {{< /tabpane >}} -Successful installation of make will show the following when the `make --version` command is run. +After Make is installed, run it to print the version. + +```bash +make --version +``` + +The output is similar to: ```output -$ make --version GNU Make 4.3 Built for x86_64-pc-linux-gnu Copyright (C) 1988-2020 Free Software Foundation, Inc. @@ -88,21 +106,23 @@ License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. ``` + {{% notice Note %}} -To run this learning path on macOS, you need to verify that your installation is for the GNU Make - not the BSD version. +If you are using macOS, you need to verify that your installation is for GNU Make - not the BSD version. You should see GNU in the version output. {{% /notice %}} -## Install Arm GNU toolchain -The toolchain is used to compile code from the host to the embedded device architecture. +### Install the Arm GNU toolchain + +The toolchain is used to compile code on the host for the embedded device architecture. {{< tabpane code=true >}} - {{< tab header="Linux" language="shell">}} + {{< tab header="x86 Linux" language="shell">}} cd $HOME wget https://developer.arm.com/-/media/Files/downloads/gnu/13.2.rel1/binrel/arm-gnu-toolchain-13.2.rel1-x86_64-arm-none-eabi.tar.xz tar -xvf arm-gnu-toolchain-13.2.rel1-x86_64-arm-none-eabi.tar.xz export PATH="$HOME/arm-gnu-toolchain-13.2.Rel1-x86_64-arm-none-eabi/bin/:$PATH" {{< /tab >}} - {{< tab header="MacOS" language="shell">}} + {{< tab header="macOS" language="shell">}} cd $HOME wget https://developer.arm.com/-/media/Files/downloads/gnu/13.3.rel1/binrel/arm-gnu-toolchain-13.3.rel1-darwin-arm64-arm-none-eabi.tar.xz tar -xvf arm-gnu-toolchain-13.3.rel1-darwin-arm64-arm-none-eabi.tar.xz @@ -111,8 +131,8 @@ export PATH="$HOME/arm-gnu-toolchain-13.3.rel1-darwin-arm64-arm-none-eabi/bin/:$ {{< /tabpane >}} {{% notice %}} -You can add the `export` command to the `.bashrc` file. This was, the Arm GNU toolchain is configured from new terminal sessions as well. +You can add the `export` command to your `.bashrc` or `.zshrc` file to set the search path for each new shell. {{% /notice %}} -Now that your development environment is set up, move on to the next section where you will generate the firmware image. \ No newline at end of file +Now that your development environment is ready, move to the next section where you will generate the firmware image. \ No newline at end of file diff --git a/content/learning-paths/microcontrollers/yolo-on-himax/flash-and-run.md b/content/learning-paths/microcontrollers/yolo-on-himax/flash-and-run.md index 78d03dcf5b..43892699b7 100644 --- a/content/learning-paths/microcontrollers/yolo-on-himax/flash-and-run.md +++ b/content/learning-paths/microcontrollers/yolo-on-himax/flash-and-run.md @@ -6,11 +6,13 @@ weight: 4 layout: learningpathall --- -Now that you have generated an image file on the local host machine, you are ready to flash the microcontroller with this firmware. +Now that you have generated an image file on the local host machine, you are ready to flash the microcontroller with the firmware. ## Install xmodem -`Xmodem` is a basic file transfer protocol which is easily installed using the Himax examples repository. Run the following command to install the dependency. If you cloned the repository to a different location, replace $HOME with the path. +`Xmodem` is a basic file transfer protocol which is easily installed using the Himax examples repository. + +Run the following command to install the dependency: ```bash cd $HOME/Seeed_Grove_Vision_AI_Module_V2 @@ -19,11 +21,13 @@ pip install -r xmodem/requirements.txt ## Connect the module -To prepare for the next steps, it's time to get the board set up. Insert the Flexible printed circuit (FPC) into the Grove Vision AI V2 module. Lift the dark grey latch on the connector as per the image below. +It's time to get the board set up. + +Insert the Flexible printed circuit (FPC) into the Grove Vision AI V2 module. Lift the dark grey latch on the connector as per the image below. ![unlatched](./unlatched.jpg) -Then, slide the FPC connector in with the metal pins facing down and close the dark grey latch to fasten the connector. +Slide the FPC connector in with the metal pins facing down and close the dark grey latch to fasten the connector. ![latched](./latched.jpg) @@ -56,11 +60,11 @@ chmod 0777 ``` {{% /notice %}} -The full path to the port is needed in the next step, so be sure to note it down. +The full path to the port is needed in the next step, so be sure to save it. ## Flash the firmware onto the module -Run the python script below to flash the firmware. +Run the python script below to flash the firmware: ```bash python xmodem\xmodem_send.py --port= \ @@ -72,7 +76,7 @@ python xmodem\xmodem_send.py --port= \ When you run other example models demonstrated in the later section [Run additional models in the web toolkit](/learning-paths/microcontrollers/yolo-on-himax/web-toolkit/), you need to adapt this command with `--model` argument. {{% /notice %}} -After the firmware image burning is completed, the message `Do you want to end file transmission and reboot system? (y)` is displayed. Press the reset button indicated in the image below. +After the firmware image flashing is completed, the message `Do you want to end file transmission and reboot system? (y)` is displayed. Press the reset button shown in the image below. ![reset button](./reset_button.jpg) diff --git a/content/learning-paths/microcontrollers/yolo-on-himax/web-toolkit.md b/content/learning-paths/microcontrollers/yolo-on-himax/web-toolkit.md index f3ec67106b..e03500d02a 100644 --- a/content/learning-paths/microcontrollers/yolo-on-himax/web-toolkit.md +++ b/content/learning-paths/microcontrollers/yolo-on-himax/web-toolkit.md @@ -8,15 +8,19 @@ layout: learningpathall In this section, you will view a live camera feed with a computer vision application running. -## Modify the Makefile +## Modify the makefile -Change the directory to the where the Makefile is located. If you cloned the repository to a different location, replace $HOME with the path. +Change to the directory where the `makefile` is located. ```bash cd $HOME/Seeed_Grove_Vision_AI_Module_V2/EPII_CM55M_APP_S/ ``` -The table shows the different options available to use with the web toolkit. Modify the `APP_TYPE` field in the `makefile` to one of the values in the table. Then pass the `--model` argument to the python `xmodem` command. +The table shows the different options available to use with the web toolkit. + +Use a text editor to open `makefile` and modify the `APP_TYPE` field to one of the values in the table. + +You will use the model argument value with the `--model` option to the `xmodem` command. |APP_TYPE |Description | Model argument | |--- |--- |--- @@ -28,22 +32,22 @@ For `tflm_fd_fm`, you need to pass all three models as separate `--model` argume {{% /notice %}} - ## Regenerate the firmware image -Now you can run `make` to re-generate the `.elf` file. +Use Make to re-generate the `.elf` file. ```bash make clean make ``` -Use the commands from [Flash firmware onto the microcontroller](/learning-paths/microcontrollers/yolo-on-himax/flash-and-run/) section to run re-generate the firmware image. +Use the same commands from previous section to run re-generate the firmware image: ```bash cd ../we2_image_gen_local/ cp ../EPII_CM55M_APP_S/obj_epii_evb_icv30_bdv10/gnu_epii_evb_WLCSP65/EPII_CM55M_gnu_epii_evb_WLCSP65_s.elf input_case1_secboot/ ``` + Run the script corresponding to the OS of your host machine. {{< tabpane code=true >}} @@ -76,11 +80,19 @@ wget https://github.com/HimaxWiseEyePlus/Seeed_Grove_Vision_AI_Module_V2/release unzip Himax_AI_web_toolkit.zip ``` -Open the unzipped directory in your file browsing system and double click `index.html`. This will open the GUI within your default browser. +{{% notice Note %}} +If needed, install the unzip command: + +```bash +sudo apt install unzip -y +``` +{{% /notice %}} + +Open the file `index.html` in a browser. You can double click the file in a file browser or use the `File -> Open File...` command from the browser menu. ## Connect to the Grove Vision AI -Select `Grove Vision AI(V2)` in the top-right hand corner and press `Connect` button. Follow the instructions to set up the connection. Now you should see a video feed with a bounding box showing identified objects, poses or face detection. +Select `Grove Vision AI(V2)` in the top-right hand corner and press `Connect` button. Follow the instructions to set up the connection. You should see a video feed with a bounding box showing identified objects, poses or face detection. ![Himax web UI](./himax_web_ui.jpg)