Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions content/learning-paths/microcontrollers/yolo-on-himax/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,18 +3,18 @@ title: Run a Computer Vision Model on a Himax Microcontroller

minutes_to_complete: 90

who_is_this_for: This is an introduction topic for beginners on how to run a computer vision application on an embedded device from Himax. This example uses an off-the-shelf Himax WiseEye2 module which is based on the Arm Cortex-M55 and Ethos-U55.
who_is_this_for: This is an introduction topic explaining how to run a computer vision application on an embedded device from Himax. The example uses an off-the-shelf Himax WiseEye2 module which is based on Arm Cortex-M55 and Ethos-U55.

learning_objectives:
- Run a you-only-look-once (YOLO) object detection model on the edge device
- Build the Himax Software Development Kit (SDK) and generate the firmware image file
- Update the firmware on the edge device (Himax WiseEye2)
- Run a you-only-look-once (YOLO) object detection model on the Himax device.
- Build the Himax Software Development Kit (SDK) and generate the firmware image file.
- Update the firmware on the Himax WiseEye2.

prerequisites:
- A [Seeed Grove Vision AI Module V2](https://www.seeedstudio.com/Grove-Vision-AI-Module-V2-p-5851.html) development board
- A [OV5647-62 Camera Module](https://www.seeedstudio.com/OV5647-69-1-FOV-Camera-module-for-Raspberry-Pi-3B-4B-p-5484.html) and included FPC cable
- A USB-C cable
- An x86 based Linux machine or a machine running Apple Silicon
- A [Seeed Grove Vision AI Module V2](https://www.seeedstudio.com/Grove-Vision-AI-Module-V2-p-5851.html) development board.
- An [OV5647-62 Camera Module](https://www.seeedstudio.com/OV5647-69-1-FOV-Camera-module-for-Raspberry-Pi-3B-4B-p-5484.html) and included FPC cable.
- A USB-C cable.
- An x86 Linux machine or a Mac running macOS with Apple Silicon.

author_primary: Chaodong Gong, Alex Su, Kieran Hejmadi

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,15 @@ weight: 3
layout: learningpathall
---

This section will walk you though the process of generating the firmware image file.
This section explains the process of generating a firmware image file.

## Clone the Himax project
## Clone the Himax GitHub project

Himax has set up a repository containing a few examples for the Seeed Grove Vision AI V2 board. It contains third-party software and scripts to build and flash the image with the object detection application. By recursively cloning the Himax examples repo, git will include the necessary sub-repositories that have been configured for the project.
Himax maintains a repository containing a few examples for the Seeed Grove Vision AI V2 board.

It contains third-party software and scripts to build and flash the image with the object detection application. By recursively cloning the Himax examples repo, git will include the necessary sub-repositories that have been configured for the project.

Clone the repository:

```bash
git clone --recursive https://github.com/HimaxWiseEyePlus/Seeed_Grove_Vision_AI_Module_V2.git
Expand All @@ -19,41 +23,41 @@ cd Seeed_Grove_Vision_AI_Module_V2

## Compile the firmware

For the object detection to activate, you need to edit the project's `makefile`, located in the `EPII_CM55M_APP_S` directory.
Use Make to compile the source code for object detection.

```bash
cd EPII_CM55M_APP_S
```

Use the `make` build tool to compile the source code. This should take up to 10 minutes depending on the number of CPU cores available on your host machine. The result is an `.elf` file written to the directory below.
This takes up to 10 minutes depending on the number of CPU cores available on your host machine.

```bash
make clean
cd EPII_CM55M_APP_S
make
```

When the build is complete, you have an `.elf` file at `obj_epii_evb_icv30_bdv10/gnu_epii_evb_WLCSP65/EPII_CM55M_gnu_epii_evb_WLCSP65_s.elf`

## Generate the firmware image

The examples repository contains scripts to generate the image file. Copy the `.elf` file to the `input_case1_secboot` directory.
The examples repository contains scripts to generate the image file.

Copy the `.elf` file to the `input_case1_secboot` directory.

```bash
cd ../we2_image_gen_local/
cp ../EPII_CM55M_APP_S/obj_epii_evb_icv30_bdv10/gnu_epii_evb_WLCSP65/EPII_CM55M_gnu_epii_evb_WLCSP65_s.elf input_case1_secboot/
```

Run the script corresponding to the OS of your host machine. This will create a file named `output.img` in the `output_case1_sec_wlcsp` directory.
Run the script your OS as shown below. This will create a file named `output.img` in the `output_case1_sec_wlcsp` directory.


{{< tabpane code=true >}}
{{< tab header="Linux" language="shell">}}
./we2_local_image_gen project_case1_blp_wlcsp.json
{{< /tab >}}
{{< tab header="MacOS" language="shell">}}
{{< tab header="macOS" language="shell">}}
./we2_local_image_gen_macOS_arm64 project_case1_blp_wlcsp.json
{{< /tab >}}
{{< /tabpane >}}

Your terminal output should end with the following.
The script output ends with the following output:

```output
Output image: output_case1_sec_wlcsp/output.img
Expand All @@ -62,4 +66,4 @@ Output image: output_case1_sec_wlcsp/output.img
IMAGE GEN DONE
```

With this step, you are ready to flash the image onto the Himax development board.
You are ready to flash the image onto the Himax development board.
74 changes: 47 additions & 27 deletions content/learning-paths/microcontrollers/yolo-on-himax/dev-env.md
Original file line number Diff line number Diff line change
@@ -1,48 +1,59 @@
---
title: Set up environment
title: Set up the environment
weight: 2

### FIXED, DO NOT MODIFY
layout: learningpathall
---

# Set up the development environment

This learning path has been validated on Ubuntu 22.04 LTS and macOS.
This Learning Path has been validated on Ubuntu 22.04 LTS and macOS.

{{% notice %}}
If you are running Windows on your host machine, you can use Ubuntu through Windows subsystem for Linux 2 (WSL2). Check out [this learning path](https://learn.arm.com/learning-paths/laptops-and-desktops/wsl2/setup/) to get started.
If you are running Windows, you can use Ubuntu through Windows subsystem for Linux 2 (WSL2). Check out [Get started with Windows Subsystem for Linux (WSL) on Arm](https://learn.arm.com/learning-paths/laptops-and-desktops/wsl2/setup/) to learn more.
{{% /notice %}}

## Install Python, pip and git
## Install software tools

Follow the instructions below to install the required development tools.

### Install Python and Pip

You will use Python to build the firmware image and pip to install additional dependencies.

Verify Python is installed by running:

You will use Python to build the firmware image and pip to install some dependencies. Verify Python is installed by running
```bash
python3 --version
```

You should see an output like the following.
You should see an output like the following:

```output
Python 3.12.7
```

Install `pip` and `venv` with the following commands.
On Ubuntu, you may need to install `pip` and `venv` with the following commands:

```bash
sudo apt update
sudo apt install python3-pip python3-venv -y
```

check the output to verify `pip` is installed correctly.
```
Verify Pip is installed correctly:

```bash
pip3 --version
```

The output is similar to:

```output
pip 24.2 from /<path-to>/pip (python 3.12)
pip 24.0 from /usr/lib/python3/dist-packages/pip (python 3.12)
```

It is considered good practice to manage `pip` packages through a virtual environment. Create one with the steps below.
It is good practice to manage Python packages through a virtual environment.

Create one with the steps below.

```bash
python3 -m venv $HOME/yolo-venv
Expand All @@ -51,58 +62,67 @@ source $HOME/yolo-venv/bin/activate

Your terminal displays `(yolo-venv)` in the prompt indicating the virtual environment is active.

You will need to have the git version control system installed. Run the command below to verify that git is installed on your system.
You also need the Git distributed version control system installed.

Run the command below to verify that Git is installed on your system:

```bash
git --version
```

You should see output similar to that below.
If it is installed, you will see output similar to:

```output
git version 2.39.3
```

## Install make
### Install Make

Install the make build tool, which is used to build the firmware in the next section.
Install the Make build tool, which is used to build the firmware in the next section.

{{< tabpane code=true >}}
{{< tab header="Linux" language="shell">}}
sudo apt update
sudo apt install make -y
{{< /tab >}}
{{< tab header="MacOS" language="shell">}}
{{< tab header="macOS" language="shell">}}
brew install make
{{< /tab >}}
{{< /tabpane >}}

Successful installation of make will show the following when the `make --version` command is run.
After Make is installed, run it to print the version.

```bash
make --version
```

The output is similar to:

```output
$ make --version
GNU Make 4.3
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
```

{{% notice Note %}}
To run this learning path on macOS, you need to verify that your installation is for the GNU Make - not the BSD version.
If you are using macOS, you need to verify that your installation is for GNU Make - not the BSD version. You should see GNU in the version output.
{{% /notice %}}
## Install Arm GNU toolchain

The toolchain is used to compile code from the host to the embedded device architecture.
### Install the Arm GNU toolchain

The toolchain is used to compile code on the host for the embedded device architecture.

{{< tabpane code=true >}}
{{< tab header="Linux" language="shell">}}
{{< tab header="x86 Linux" language="shell">}}
cd $HOME
wget https://developer.arm.com/-/media/Files/downloads/gnu/13.2.rel1/binrel/arm-gnu-toolchain-13.2.rel1-x86_64-arm-none-eabi.tar.xz
tar -xvf arm-gnu-toolchain-13.2.rel1-x86_64-arm-none-eabi.tar.xz
export PATH="$HOME/arm-gnu-toolchain-13.2.Rel1-x86_64-arm-none-eabi/bin/:$PATH"
{{< /tab >}}
{{< tab header="MacOS" language="shell">}}
{{< tab header="macOS" language="shell">}}
cd $HOME
wget https://developer.arm.com/-/media/Files/downloads/gnu/13.3.rel1/binrel/arm-gnu-toolchain-13.3.rel1-darwin-arm64-arm-none-eabi.tar.xz
tar -xvf arm-gnu-toolchain-13.3.rel1-darwin-arm64-arm-none-eabi.tar.xz
Expand All @@ -111,8 +131,8 @@ export PATH="$HOME/arm-gnu-toolchain-13.3.rel1-darwin-arm64-arm-none-eabi/bin/:$
{{< /tabpane >}}

{{% notice %}}
You can add the `export` command to the `.bashrc` file. This was, the Arm GNU toolchain is configured from new terminal sessions as well.
You can add the `export` command to your `.bashrc` or `.zshrc` file to set the search path for each new shell.
{{% /notice %}}


Now that your development environment is set up, move on to the next section where you will generate the firmware image.
Now that your development environment is ready, move to the next section where you will generate the firmware image.
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,13 @@ weight: 4
layout: learningpathall
---

Now that you have generated an image file on the local host machine, you are ready to flash the microcontroller with this firmware.
Now that you have generated an image file on the local host machine, you are ready to flash the microcontroller with the firmware.

## Install xmodem

`Xmodem` is a basic file transfer protocol which is easily installed using the Himax examples repository. Run the following command to install the dependency. If you cloned the repository to a different location, replace $HOME with the path.
`Xmodem` is a basic file transfer protocol which is easily installed using the Himax examples repository.

Run the following command to install the dependency:

```bash
cd $HOME/Seeed_Grove_Vision_AI_Module_V2
Expand All @@ -19,11 +21,13 @@ pip install -r xmodem/requirements.txt

## Connect the module

To prepare for the next steps, it's time to get the board set up. Insert the Flexible printed circuit (FPC) into the Grove Vision AI V2 module. Lift the dark grey latch on the connector as per the image below.
It's time to get the board set up.

Insert the Flexible printed circuit (FPC) into the Grove Vision AI V2 module. Lift the dark grey latch on the connector as per the image below.

![unlatched](./unlatched.jpg)

Then, slide the FPC connector in with the metal pins facing down and close the dark grey latch to fasten the connector.
Slide the FPC connector in with the metal pins facing down and close the dark grey latch to fasten the connector.

![latched](./latched.jpg)

Expand Down Expand Up @@ -56,11 +60,11 @@ chmod 0777 <COM port>
```
{{% /notice %}}

The full path to the port is needed in the next step, so be sure to note it down.
The full path to the port is needed in the next step, so be sure to save it.

## Flash the firmware onto the module

Run the python script below to flash the firmware.
Run the python script below to flash the firmware:

```bash
python xmodem\xmodem_send.py --port=<COM port> \
Expand All @@ -72,7 +76,7 @@ python xmodem\xmodem_send.py --port=<COM port> \
When you run other example models demonstrated in the later section [Run additional models in the web toolkit](/learning-paths/microcontrollers/yolo-on-himax/web-toolkit/), you need to adapt this command with `--model` argument.
{{% /notice %}}

After the firmware image burning is completed, the message `Do you want to end file transmission and reboot system? (y)` is displayed. Press the reset button indicated in the image below.
After the firmware image flashing is completed, the message `Do you want to end file transmission and reboot system? (y)` is displayed. Press the reset button shown in the image below.

![reset button](./reset_button.jpg)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,19 @@ layout: learningpathall

In this section, you will view a live camera feed with a computer vision application running.

## Modify the Makefile
## Modify the makefile

Change the directory to the where the Makefile is located. If you cloned the repository to a different location, replace $HOME with the path.
Change to the directory where the `makefile` is located.

```bash
cd $HOME/Seeed_Grove_Vision_AI_Module_V2/EPII_CM55M_APP_S/
```

The table shows the different options available to use with the web toolkit. Modify the `APP_TYPE` field in the `makefile` to one of the values in the table. Then pass the `--model` argument to the python `xmodem` command.
The table shows the different options available to use with the web toolkit.

Use a text editor to open `makefile` and modify the `APP_TYPE` field to one of the values in the table.

You will use the model argument value with the `--model` option to the `xmodem` command.

|APP_TYPE |Description | Model argument |
|--- |--- |---
Expand All @@ -28,22 +32,22 @@ For `tflm_fd_fm`, you need to pass all three models as separate `--model` argume
{{% /notice %}}



## Regenerate the firmware image

Now you can run `make` to re-generate the `.elf` file.
Use Make to re-generate the `.elf` file.

```bash
make clean
make
```

Use the commands from [Flash firmware onto the microcontroller](/learning-paths/microcontrollers/yolo-on-himax/flash-and-run/) section to run re-generate the firmware image.
Use the same commands from previous section to run re-generate the firmware image:

```bash
cd ../we2_image_gen_local/
cp ../EPII_CM55M_APP_S/obj_epii_evb_icv30_bdv10/gnu_epii_evb_WLCSP65/EPII_CM55M_gnu_epii_evb_WLCSP65_s.elf input_case1_secboot/
```

Run the script corresponding to the OS of your host machine.

{{< tabpane code=true >}}
Expand Down Expand Up @@ -76,11 +80,19 @@ wget https://github.com/HimaxWiseEyePlus/Seeed_Grove_Vision_AI_Module_V2/release
unzip Himax_AI_web_toolkit.zip
```

Open the unzipped directory in your file browsing system and double click `index.html`. This will open the GUI within your default browser.
{{% notice Note %}}
If needed, install the unzip command:

```bash
sudo apt install unzip -y
```
{{% /notice %}}

Open the file `index.html` in a browser. You can double click the file in a file browser or use the `File -> Open File...` command from the browser menu.

## Connect to the Grove Vision AI

Select `Grove Vision AI(V2)` in the top-right hand corner and press `Connect` button. Follow the instructions to set up the connection. Now you should see a video feed with a bounding box showing identified objects, poses or face detection.
Select `Grove Vision AI(V2)` in the top-right hand corner and press `Connect` button. Follow the instructions to set up the connection. You should see a video feed with a bounding box showing identified objects, poses or face detection.

![Himax web UI](./himax_web_ui.jpg)

Expand Down