Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GroundedSAM labels #6

Open
csv610 opened this issue Oct 30, 2023 · 13 comments
Open

GroundedSAM labels #6

csv610 opened this issue Oct 30, 2023 · 13 comments
Assignees

Comments

@csv610
Copy link

csv610 commented Oct 30, 2023

Hello,

When I look at the labels generated by GroundedSAM, I see a huge list of floating number not the data bounding boxes
in yolo format. How should I interpret the output? Is it possible to store the output inn the Yolo format?

@capjamesg
Copy link
Member

Can you share an example of the output?

@capjamesg capjamesg self-assigned this Nov 1, 2023
@shersoni610
Copy link

Hi, This is the label for an image containing a single object (sharp edges).

0 0.41200 0.09202 0.41100 0.09313 0.41100 0.09424 0.40800 0.09756 0.40800 0.09978 0.40700 0.10089 0.40700 0.10200 0.40600 0.10310 0.40600 0.10421 0.40500 0.10532 0.40500 0.10643 0.40400 0.10754 0.40400 0.10865 0.40300 0.10976 0.40300 0.11419 0.40200 0.11530 0.40200 0.11641 0.40000 0.11863 0.40000 0.12084 0.39900 0.12195 0.39900 0.12417 0.39800 0.12528 0.39800 0.12639 0.39700 0.12749 0.39700 0.12860 0.39600 0.12971 0.39600 0.13304 0.39500 0.13415 0.39500 0.13858 0.39600 0.13969 0.39500 0.14080 0.39500 0.15965 0.39400 0.16075 0.39400 0.17627 0.39300 0.17738 0.39100 0.17517 0.39100 0.17295 0.39200 0.17184 0.39200 0.17073 0.39100 0.16962 0.39100 0.16297 0.39000 0.16186 0.39000 0.16075 0.38800 0.16075 0.38700 0.15965 0.38600 0.15965 0.38500 0.15854 0.37900 0.15854 0.37800 0.15965 0.37300 0.15965 0.37200 0.16075 0.36700 0.16075 0.36600 0.16186 0.36300 0.16186 0.36200 0.16297 0.35700 0.16297 0.35600 0.16408 0.35100 0.16408 0.35000 0.16519 0.34800 0.16519 0.34700 0.16630 0.34500 0.16630 0.34400 0.16741 0.34000 0.16741 0.33900 0.16851 0.33600 0.16851 0.33500 0.16962 0.33300 0.16962 0.33200 0.17073 0.33000 0.17073 0.32900 0.17184 0.32500 0.17184 0.32400 0.17295 0.32100 0.17295 0.32000 0.17406 0.31900 0.17406 0.31800 0.17517 0.31600 0.17517 0.31500 0.17627 0.31200 0.17627 0.31100 0.17738 0.31000 0.17738 0.30900 0.17849 0.30700 0.17849 0.30600 0.17960 0.30400 0.17960 0.30300 0.18071 0.30100 0.18071 0.30000 0.18182 0.29800 0.18182 0.29700 0.18293 0.29600 0.18293 0.29500 0.18404 0.29300 0.18404 0.29200 0.18514 0.28900 0.18514 0.28800 0.18625 0.28700 0.18625 0.28600 0.18736 0.28500 0.18736 0.28400 0.18847 0.28200 0.18847 0.28100 0.18958 0.27700 0.18958 0.27600 0.19069 0.27500 0.19069 0.27400 0.19180 0.27300 0.19180 0.27200 0.19290 0.27100 0.19290 0.27000 0.19401 0.26800 0.19401 0.26700 0.19512 0.26600 0.19512 0.26500 0.19623 0.26400 0.19623 0.26300 0.19734 0.26000 0.19734 0.25900 0.19845 0.25700 0.19845 0.25600 0.19956 0.25500 0.19956 0.25400 0.20067 0.25300 0.20067 0.25200 0.20177 0.25000 0.20177 0.24900 0.20288 0.24600 0.20288 0.24500 0.20399 0.24300 0.20399 0.24100 0.20621 0.24000 0.20621 0.23900 0.20732 0.23600 0.20732 0.23500 0.20843 0.23400 0.20843 0.23300 0.20953 0.23200 0.20953 0.23100 0.21064 0.22900 0.21064 0.22800 0.21175 0.22600 0.21175 0.22500 0.21286 0.22400 0.21286 0.22300 0.21397 0.22200 0.21397 0.22100 0.21508 0.21800 0.21508 0.21700 0.21619 0.21600 0.21619 0.21500 0.21729 0.21300 0.21729 0.21200 0.21840 0.21100 0.21840 0.21000 0.21951 0.20900 0.21951 0.20800 0.22062 0.20600 0.22062 0.20500 0.22173 0.20400 0.22173 0.20300 0.22284 0.20200 0.22284 0.20100 0.22395 0.19900 0.22395 0.19800 0.22506 0.19600 0.22506 0.19400 0.22727 0.19300 0.22727 0.19200 0.22838 0.19000 0.22838 0.18900 0.22949 0.18800 0.22949 0.18600 0.23171 0.18500 0.23171 0.18400 0.23282 0.18200 0.23282 0.18100 0.23392 0.18000 0.23392 0.17800 0.23614 0.17700 0.23614 0.17600 0.23725 0.17500 0.23725 0.17400 0.23836 0.17300 0.23836 0.17100 0.24058 0.17000 0.24058 0.16900 0.24169 0.16800 0.24169 0.16600 0.24390 0.16500 0.24390 0.16300 0.24612 0.16200 0.24612 0.16000 0.24834 0.15900 0.24834 0.15700 0.25055 0.15600 0.25055 0.15400 0.25277 0.15300 0.25277 0.15100 0.25499 0.15000 0.25499 0.14600 0.25942 0.14500 0.25942 0.13800 0.26718 0.13700 0.26718 0.13500 0.26940 0.13400 0.26940 0.11500 0.29047 0.11500 0.29157 0.11200 0.29490 0.11200 0.29601 0.10900 0.29933 0.10900 0.30044 0.10300 0.30710 0.10300 0.30820 0.10100 0.31042 0.10100 0.31153 0.10000 0.31264 0.10000 0.31375 0.09700 0.31707 0.09700 0.31818 0.09500 0.32040 0.09500 0.32151 0.09300 0.32373 0.09300 0.32483 0.09200 0.32594 0.09200 0.32705 0.09100 0.32816 0.09100 0.32927 0.09000 0.33038 0.09000 0.33149 0.08900 0.33259 0.08900 0.33370 0.08700 0.33592 0.08700 0.33703 0.08600 0.33814 0.08600 0.33925 0.08500 0.34035 0.08500 0.34257 0.08300 0.34479 0.08300 0.34590 0.08200 0.34701 0.08200 0.34812 0.08100 0.34922 0.08100 0.35033 0.08000 0.35144 0.08000 0.35255 0.07900 0.35366 0.07900 0.35477 0.07800 0.35588 0.07800 0.35698 0.07700 0.35809 0.07700 0.35920 0.07600 0.36031 0.07600 0.36142 0.07500 0.36253 0.07500 0.36475 0.07400 0.36585 0.07400 0.36696 0.07300 0.36807 0.07300 0.36918 0.07200 0.37029 0.07200 0.37140 0.07100 0.37251 0.07100 0.37361 0.07000 0.37472 0.07000 0.37805 0.06900 0.37916 0.06900 0.38027 0.06800 0.38137 0.06800 0.38359 0.06700 0.38470 0.06700 0.38581 0.06600 0.38692 0.06600 0.38914 0.06500 0.39024 0.06500 0.39246 0.06400 0.39357 0.06400 0.39579 0.06300 0.39690 0.06300 0.39911 0.06200 0.40022 0.06200 0.40244 0.06100 0.40355 0.06100 0.40576 0.06000 0.40687 0.06000 0.40798 0.05900 0.40909 0.05900 0.41463 0.05800 0.41574 0.05800 0.42018 0.05700 0.42129 0.05700 0.42572 0.05600 0.42683 0.05600 0.43348 0.05500 0.43459 0.05500 0.43792 0.05400 0.43902 0.05400 0.45011 0.05300 0.45122 0.05300 0.45898 0.05200 0.46009 0.05200 0.47118 0.05100 0.47228 0.05100 0.48448 0.05000 0.48559 0.05000 0.50222 0.05100 0.50333 0.05100 0.52439 0.05200 0.52550 0.05200 0.53437 0.05300 0.53548 0.05300 0.54435 0.05400 0.54545 0.05400 0.55543 0.05500 0.55654 0.05500 0.56319 0.
"Image_1.txt" 1L, 17458B

@shersoni610
Copy link

I was hoping GroundedSAM to give a bounding box.

@capjamesg
Copy link
Member

Grounded SAM is a hybrid model that runs Grounding DINO first then SAM, a segmentation model. The output is thus in segmentation form. You can load the values after the first 0 (the class ID) into supervision to get bounding boxes. See https://blog.roboflow.com/convert-bboxes-masks-polygons/#how-to-convert-a-mask-to-bounding-box-mask-to-xyxy for more information.

@capjamesg
Copy link
Member

If you want exclusively bounding boxes, autodistill-grounding-dino might be more appropriate (it is also faster).

@csv610
Copy link
Author

csv610 commented Nov 3, 2023 via email

@capjamesg
Copy link
Member

You can train a YOLOv8 segmentation model with Autodistill. See https://github.com/autodistill/autodistill-yolov8?tab=readme-ov-file#choosing-a-task for guidance on how to specify that you want to train a segmentation model. You will need to choose a base model that supported segmentation, like Grounded SAM or FastSAM. See https://github.com/autodistill/autodistill?tab=readme-ov-file#-available-models for more information about supported models.

@csv610
Copy link
Author

csv610 commented Nov 3, 2023 via email

@capjamesg
Copy link
Member

In that case, I recommend Autodistill Grounding DINO. Grounding DINO is a zero-shot object detection model that you can use to label images with bounding boxes.

@csv610
Copy link
Author

csv610 commented Nov 5, 2023 via email

@capjamesg
Copy link
Member

Good question! Every model has its strengths and weaknesses. We prioritize implementing state-of-the-art models. We expect the library of models to grow as the state of the art gets better.

We encourage people to try different models to see which one works well for their use case. For example, OWLv2 and Grounding DINO are both impressive models capable of identifying a range of objects, but there isn't an objective "best". We often. recommend Grounding DINO as a starting point for object detection, but we haven't noted this in our documentation. We are discussing how best to accomplish this.

@dilwolf
Copy link

dilwolf commented Apr 30, 2024

If you want exclusively bounding boxes, autodistill-grounding-dino might be more appropriate (it is also faster).

Thanks a lot for saving my time! Since it is my first time using autodistill, I had the issue that converting polygon to bbox. Because the getting bbox was just enough for my case. I used autodistill-grounding-dino. This suggestion really helped.

@dilwolf
Copy link

dilwolf commented Apr 30, 2024

If you want exclusively bounding boxes, autodistill-grounding-dino might be more appropriate (it is also faster).

By the way, the accuracy of autodistill-grounding-dino is the same as the accuracy of autodistill-grounding-sam for object detection, right?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants