Requesting feedback on YOLOv9/EdgeTPU Google Coral Plugin

[dbro]

Hello Frigate community,
https://github.com/dbro/frigate-detector-edgetpu-yolo9 is a new and separate repository that adds support for Frigate to run YOLO models (v9, maybe others) on Google Coral EdgeTPU hardware.

It can run a YOLO v9 model at 10ms on a Google Coral mini-PCIe card. The model is the "t" version with dimensions 192x192, trained using only COCO data.

Why a separate repository instead of a pull request? This approach follows the suggestion from a previous issue discussion (#15630, this comment) to avoid modifying Frigate's code base and instead to provide a custom detector plugin in a separate codebase which can have a different license that is compatible with YOLO's license (issue #16335 talks about this).

The code is based on Frigate's existing edgetpu-tfl.py detector, with post-processing steps adapted from code developed about 2 years ago (see #7650) by users such as @lkaino and @harakas (Thank you!).

The audience for this are people who have Google Coral and are getting too many false positive detections from the default EdgeTPU model (MobileDet). This approach is slightly less efficient: its detection speed is about 10ms vs the 7ms from MobileDet, and there is a small amount of CPU used for post-processing that is not necessary with MobileDet. In my testing, these were acceptable tradeoffs for the improvement in detection accuracy.

Comments and suggestions are welcome. In particular, should I create a pull request to mention this in the documentation page about Object Detectors?

Also if anyone has been able to get YOLO-NAS working with Google Coral, or has found a place to download the INT-8 versions of YOLO-NAS, please let me know!

[NickM-27]

I don't think there is any major issue with this, a few things I am curious about and one suggestion for the PR:

1. Is the model quantized? I believe it would have to be to run on the coral but I am not aware of quantization support for YOLOv9. How was the model created?
2. 192x192 is quite small, what were the speeds at 320x320?
3. Regarding the PR, I would suggest looking at the existing code, many of the post processing functions might be handled by the existing model utilty

[hawkeye217]

Are you sure this is working correctly? Your documentation notes that users should add model: under the detectors: section in the config, which is not correct.

[NickM-27]

Oh, I see this is based off of Ultralytics YOLOv9, not https://github.com/WongKinYiu/yolov9

In that case I don't believe this is something we can accept, see #10717 for reference

[dbro]

Thanks for the feedback, here are some responses.

Yes, the model is quantized by the ultralytics software that runs in Google Colab that translates the original .pt model file.

The speed for a 320x320 size model was 30ms, and it puts more load on the CPU which I want to avoid because my system tends to overheat when there are a lot (50/sec ?) of detections that involve this much CPU activity. Is there a way to limit the detections/second, to cause frigate to throttle the requests going to the Coral TPU?

Thanks for the suggestion to use existing model utilities, I will check it out.

Sorry for the sloppy description, I should have said "after the detectors: section" instead of "below" which implies a child relationship.

Yes, I saw that code relating to yolov8 was removed from frigate in March 2024. There was a suggestion made in December 2024 by @blakeblackshear that it would be acceptable to do this with a separate repo.

I can look into using the alternate YOLO v9 that you linked to. I'll check this out too.

Thanks again

[dbro]

Update: the latest version of this repository uses a different source for the YOLO model that is compatible with Frigate's license. Information about how the model is built, quantized, and converted is in this issue dbro/frigate-detector-edgetpu-yolo9#1 (comment)