HAILO Bottlecap detection Demo

Introduction

This page details the process for getting a .hef model to run on a HAILO8L TPU, as well as running actual object detection code with our custom .hef model for detecting bottle caps

Downloading Prerequisites

You need a 3.10+ Version of Python installed on your system. For our Demo we used python3.13

As well as git, git-lfs, python3-pip, python3-venv and virtualenv:

sudo apt update
sudo apt install git git-lfs python3-pip python3-venv virtualenv

1. Make an Account in the Hailo developer zone
2. Go to Software Downloads and select sub package hailort, x86 (or arm), Hailo8/L, and Python3.13 for the filter
3. Then download:

• HailoRT – PCIe driver Ubuntu package (deb),
• HailoRT – Python package (whl) for Python 3.13, x86_64,
• HailoRT – Ubuntu package (deb) for amd64

Transfer these over to the device where the hailo chip is installed in.

If you want to quantize the existing model yourself, then you also need to download the Hailo AI Software Suite and docker. The Process takes a long time and requires a powerful pc so be warned.

Install driver and hailort library for Hailo8/8L

sudo dpkg -i hailort_4.23.0_amd64.deb
sudo dpkg -i hailort-pcie-driver_4.23.0_all.deb
# Reboot after Installation
sudo reboot

After Installation

Verify the hailortcli can find the hailo device:

hailortcli scan

The result should look like this:

Hailo Devices:
[-] Device: 0000:03:00.0

Testing the Object Detection Model

The following commands download the model,training data and sets up the labels.txt for later use. At the end the model is benchmarked on the hailo8L giving a rough performance estimate.

# This repository contains a pretrained .onnx model and an already quantized .hef model
# that can be directly used with the hailo itself
# The Repo also contains information on how to quantize the model for the hailo8L chip
git clone https://github.com/qitechgmbh/bottle-cap-model-training
cd bottle-cap-model-training
git-lfs pull

# Copy the pretrained model to /tmp
cp yolo_models/bottle_cap_detection/weights/best.hef /tmp/best.hef
echo "bottlecap" > /tmp/labels.txt
cp caps.mov /tmp/caps.mov
cd ..

# Verify the model is functional
hailortcli benchmark /tmp/best.hef

This snippet downloads a modified version of hailo-apps and sets up a virtual environment, in which we will install our dependencies.

git clone https://github.com/qitechgmbh/hailo-apps
cd hailo-apps
virtualenv venv_hailo_apps
. setup_env.sh

Now that we are in the virtual environment we can install the python bindings for the hailort library:

# Replace with your actual path
pip install hailort-4.23.0-cp313-cp313-linux_x86_64.whl

After installing the hailort dependency we also need the dependencies for running our python example code:

cd hailo_apps/python/standalone_apps/object_detection
pip install -r requirements.txt

Now for our "bottlecap detection" we have multiple options, as the script accepts images,videos,camera feeds and more. HAILO_MONITOR=1 can be added to the environment to enable monitoring of the hailo chip usage for specific processes. Using a Camera:

HAILO_MONITOR=1 python3 object_detection.py --hef /tmp/best.hef --labels /tmp/labels.txt -i /dev/video0 --track

Using a Video file:

HAILO_MONITOR=1 python3 object_detection.py --hef /tmp/best.hef --labels /tmp/labels.txt -i /tmp/caps.mov --track

To see the live video feed go to: http://127.0.0.1:5000/ in your browser, it will be slightly delayed though.

Alternatively if no firewall is in the way you can also use the Local ip address of the PC with the Hailo chip like http://192.168.2.111:5000/ for example.

In the video feed you can see the confidence score assigned and its id, right now a bottlecap is counted if its confidence is above 85%. Due to all the training data happening on a white conveyer belt, backgrounds that are radically different may influence results.