Berries Detector

ROS 1 package for berry detection on UAV imagery, with GPS-linked outputs for mapping and visualization.

The current default runtime path uses a TensorFlow Lite model compiled for Google Coral Edge TPU and publishes per-image cloudberry counts together with georeferenced visualization markers.

What the Package Does

• Subscribes to camera images, GPS fixes, range sensor data, base-station RTK data, and camera info.
• Splits each incoming image into a grid of crops.
• Runs inference on each crop and sums the detected cloudberries.
• Publishes a sensor_msgs/NavSatFix result for each processed image.
• Publishes a visualization_msgs/Marker for RViz-style spatial visualization.

Default Runtime

The default launch file is launch/berry_detector.launch and starts:

• Node: image_gps_node_tf_lite_coral.py
• ROS package identifier: berries_detector
• Default model: models/best_full_integer_quant_edgetpu.tflite
• Namespace: $(arg UAV_NAME)

This is the active path to document and use first. The repository also contains older ONNX and non-Coral variants for comparison and experimentation.

Repository Layout

• launch/berry_detector.launch: main launch entrypoint
• config/berry_detector.yaml: runtime parameters and topic names
• scripts/image_gps_node_tf_lite_coral.py: current Coral / TFLite detector
• scripts/image_gps_node_tf_lite.py: CPU TFLite variant
• scripts/image_gps_node.py: ONNX variant
• models/: packaged model artifacts

Runtime Inputs and Outputs

Subscribed Topics

From config/berry_detector.yaml and the active Coral node:

• continuous_gopro_photo/image_raw
• continuous_gopro_photo/camera_info
• gps/filtered
• hw_api/rtk
• hw_api/distance_sensor

When launched under a UAV namespace, these resolve relative to /<UAV_NAME>/... except for topics that are configured with absolute names.

Published Topics

• inference_results as sensor_msgs/NavSatFix
• marker/berry_detection as visualization_msgs/Marker

Result Encoding

The active node stores the berry count in NavSatFix.position_covariance[0].

• Latitude, longitude, and altitude come from the closest GPS fix to the image timestamp.
• The marker position is derived relative to the RTK base station.
• The marker color is a heatmap driven by the detected cloudberry count.

Detection Logic

• Images are divided into a grid of patches based on image resolution.
• Each patch is resized to the model input size.
• The Coral/TFLite model is run per patch.
• Counts are accumulated across the full image.
• The active post-processing path counts class IDs 2 and 3, corresponding to cloudberry and cloudberry-unripe.

Default class labels are:

• Bilberry
• Bilberry-unripe
• Cloudberry
• Cloudberry-unripe
• Crowberry
• Crowberry-unripe
• Lingonberry
• Lingonberry-unripe
• Bog-bilberry
• Bog-bilberry-unripe
• Mushroom

Requirements

ROS

This is a ROS 1 catkin package. The package metadata currently declares:

• geometry_msgs
• roscpp
• rospy
• sensor_msgs
• std_msgs

At runtime, the active detector code also imports and uses:

• cv_bridge
• visualization_msgs
• mrs_msgs

Python

The repository ships requirements.txt with:

• numpy==1.24.4
• onnxruntime
• opencv-python
• cv-bridge
• pandas
• tflite_runtime

The active Coral path additionally requires:

• pycoral
• utm

For the default launch path you also need a working Coral Edge TPU runtime and a compatible Edge TPU-compiled .tflite model.

Setup

Clone the package into a ROS 1 catkin workspace:

cd ~/catkin_ws/src
git clone <repo-url> berries_detector
cd ..
catkin_make
source devel/setup.bash

Install Python dependencies in the environment used by your ROS nodes:

pip install -r src/berries_detector/requirements.txt
pip install pycoral utm

Running

Set the UAV namespace and launch the detector:

export UAV_NAME=uav15
roslaunch berries_detector berry_detector.launch

To override the model at launch time:

roslaunch berries_detector berry_detector.launch model_name:=yolo11a_edgetpu.tflite

Important Parameters

Key parameters loaded from config/berry_detector.yaml:

• PARQUET_FILE: /tmp/inference_results.parquet
• PUBLISHED_TOPIC: inference_results
• MODEL_INPUT_SIZE: [320, 320]
• IMAGE_TOPIC: continuous_gopro_photo/image_raw
• CAMERA_INFO_TOPIC: continuous_gopro_photo/camera_info
• FRAME_ID: common_origin
• MARKER_TOPIC: marker/berry_detection
• GPS_TOPIC: gps/filtered
• BASE_STATION_TOPIC: hw_api/rtk
• GARMIN_TOPIC: hw_api/distance_sensor
• ODOM_TOPIC: lio_sam/imupreintegration/odom
• MARKER_FRAME_ID: uav15/fcu
• CLOUD_BERRY_CLASS_ID: 2
• CONFIDENCE_THRESHOLD: 0.7
• CONCURRENT_WORKERS: 1

The launch file also sets:

• HFOV: 156.0
• VFOV: 156.0
• Z_REF: 0.32
• A_REF: 0.0039

Notes

• The repository includes multiple detector implementations. The launch file currently selects the Coral/TFLite path.
• The ONNX and CPU TFLite scripts are still present, but they are not the default runtime entrypoint.
• A Parquet output path is configured, but the active Coral node currently publishes ROS outputs and does not actively append inference rows to the Parquet file in its main processing path.