Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add YOLOv8, YOLOv9, and YOLO-World to FiftyOne Model Zoo #4153

Merged
merged 8 commits into from Mar 14, 2024

Conversation

jacobmarks
Copy link
Contributor

With our Ultralytics integration, it is currently possible to load a YOLOv8, YOLOv9, or YOLO-World model from Ultralytics and then apply it to a FiftyOne dataset. This PR goes a step further and directly adds these models to the model zoo.

One upside of this approach is that model location is managed by FiftyOne, as opposed to loading directly from Ultralytics, which results in the checkpoint being downloaded to your current working directory.

The PR updates the Ultralytics integration docs to reflect these new changes.

(Details)

Release Notes

Is this a user-facing change that should be mentioned in the release notes?

Adds YOLOv8, YOLOv9, and YOLO-World models directly to the zoo

  • No. You can skip the rest of this section.
  • Yes. Give a description of this change to be included in the release
    notes for FiftyOne users.

(Details in 1-2 sentences. You can just refer to another PR with a description
if this PR is part of a larger change.)

What areas of FiftyOne does this PR affect?

  • App: FiftyOne application changes
  • Build: Build and test infrastructure changes
  • Core: Core fiftyone Python library changes
  • Documentation: FiftyOne documentation changes
  • Other

Copy link

codecov bot commented Mar 12, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

❗ No coverage uploaded for pull request base (release/v0.23.7@fc22e8b). Click here to learn what that means.

Additional details and impacted files
@@                Coverage Diff                 @@
##             release/v0.23.7    #4153   +/-   ##
==================================================
  Coverage                   ?   16.03%           
==================================================
  Files                      ?      733           
  Lines                      ?    82012           
  Branches                   ?     1118           
==================================================
  Hits                       ?    13153           
  Misses                     ?    68859           
  Partials                   ?        0           
Flag Coverage Δ
app 16.03% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@brimoor brimoor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jacobmarks thanks for this! LMK what you think about my proposed refactor which should simplify things 👍

fiftyone/zoo/models/manifest-torch.json Outdated Show resolved Hide resolved
fiftyone/utils/ultralytics.py Outdated Show resolved Hide resolved
fiftyone/utils/ultralytics.py Outdated Show resolved Hide resolved
@jacobmarks
Copy link
Contributor Author

@brimoor just updated to reflect this approach!

@jacobmarks jacobmarks requested a review from brimoor March 13, 2024 00:10
Copy link
Contributor

@brimoor brimoor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! I simplified the implementation a bit (and made sure that all the integration examples still run before approving 😄)

@@ -230,6 +220,77 @@ manually convert Ultralytics predictions to :ref:`FiftyOne format <keypoints>`:
:alt: ultralytics-keypoints
:align: center

.. _ultralytics-open-vocabulary-object-detection:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved this to the end of the section because the previous sections all use the same dataset defined in the Setup section

},
"default_deployment_config_dict": {
"type": "fiftyone.utils.ultralytics.FiftyOneYOLODetectionModel",
"config": {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI actually no parameters are required here. load_zoo_model() automatically downloads the model and injects the path into the model_path of the config

@jacobmarks jacobmarks merged commit 86dac3d into release/v0.23.7 Mar 14, 2024
9 checks passed
@jacobmarks jacobmarks deleted the zoo-yolo-models branch March 14, 2024 04:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants