pollinator_detective:

detect and count pollinator visits from video frames

Contributors (Diana Santelia's Lab at ETH Zurich)

• Dr. Diana Santelia / Dr. Erik Solhaug: project leaders
• Ursina Baselgia: experiments and image annotation
• Hongyuan Zhang: code implementation

Working flow

1. Select region of interest (ROI) and track it with CSRT tracker

This was used since the flower may dance with wind, and to reduce the field of view

2. Detect pollinator within the ROI with Swin-L-DINO

The pretrained catergories: Bumblebees, Flies, Honeybees, Hoverfly_A, Hoverfly_B, and Wildbees

Data preparation

1. Data annotation was done with ISAT_with_segment_anything

Consider trying our tool for our image annotation

2. Data clustering and sampling with HDBSCAN

To form a more balanced traning set by sampling each cluster

Current bottleneck

• Unblanced datasets in terms of number of pollinators in each catergoreis and pixel sizes of them
• Hard to confirm if a pollinator is trully landed on the flower
• Misidentified pollinators such as ants and beetles