Detections.__getitem__ not supported for index of type <class 'numpy.ndarray'>.

[arami1]

Search before asking

• I have searched the Supervision issues and found no similar bug report.

Bug

this is a link to my notebook, please take a look
https://colab.research.google.com/drive/1nFgpu2bTKLDuFfxeSW6yzzHzIkKtQi9w?authuser=1#scrollTo=jbzTzHJW22up
I downloaded the dataset after doing annotations from roboflow and then uploaded the zip file to google drive.

Environment

torch: 1.13
cuda: cu116
roboflow: 0.2.34
supervision: 0.3.0
transformers: 4.26.1

Minimal Reproducible Example

No response

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

[github-actions]

Hello there, thank you for opening an Issue ! 🙏🏻 The team was notified and they will get back to you asap.

[arami1]

I have used the same code you provided.
My one annotation file is:
{"info":{"year":"2023","version":"1","description":"Exported from roboflow.ai","contributor":"","url":"https://public.roboflow.ai/object-detection/undefined","date_created":"2023-03-13T13:21:50+00:00"},"licenses":[{"id":1,"url":"https://creativecommons.org/licenses/by/4.0/","name":"CC BY 4.0"}],"categories":[{"id":0,"name":"labels","supercategory":"none"},{"id":1,"name":"rice leaf caterpillar","supercategory":"labels"},{"id":2,"name":"rice leaf roller","supercategory":"labels"}],"images":[{"id":0,"license":1,"file_name":"IP001000460_jpg.rf.7a6253055fc0141a270d2c2275d20f9b.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":1,"license":1,"file_name":"IP000000279_jpg.rf.16cb9316608f637b9ab411191a627c4a.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":2,"license":1,"file_name":"IP000000061_jpg.rf.1fd888279be601c1b8c0e703d2a227bd.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":3,"license":1,"file_name":"IP001000375_jpg.rf.3092d271809e96e0c409a00b98bb04ab.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":4,"license":1,"file_name":"IP001000016_jpg.rf.54b10e8843b41abbb7274d9fec3b2fbe.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":5,"license":1,"file_name":"IP001000179_jpg.rf.1897a09345d8cdab3c8e657bf2656fb0.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":6,"license":1,"file_name":"IP001000455_jpg.rf.005f5066752d3d20d153d83c1c8737a8.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":7,"license":1,"file_name":"IP000000075_jpg.rf.d139beb487b65d39bf1516d8ab548660.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":8,"license":1,"file_name":"IP000000320_jpg.rf.c755fca43e8eef8ff9df0bdb816c376d.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":9,"license":1,"file_name":"IP001000419_jpg.rf.61bd6c3cf3c3a704339ebeea92136de2.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":10,"license":1,"file_name":"IP000000378_jpg.rf.a2d06efb45af1b0fd7d3dc56c8d34e20.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":11,"license":1,"file_name":"IP001000390_jpg.rf.dfe752a3d5e42ae05ae87c4c5a7b0f0a.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":12,"license":1,"file_name":"IP000000134_jpg.rf.df0dd9906e7a8190b572c1ffc6d9c7fc.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":13,"license":1,"file_name":"IP001000070_jpg.rf.51a596f95247ded8d76830faf6cbcef6.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":14,"license":1,"file_name":"IP000000212_jpg.rf.f2490939f1b149abe8c8f617c48056a9.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"}],"annotations":[{"id":0,"image_id":0,"category_id":1,"bbox":[76,14,215.5,461],"area":99345.5,"segmentation":[],"iscrowd":0},{"id":1,"image_id":0,"category_id":1,"bbox":[344,0,269.5,469.5],"area":126530.25,"segmentation":[],"iscrowd":0},{"id":2,"image_id":1,"category_id":2,"bbox":[63,126,526.5,467],"area":245875.5,"segmentation":[],"iscrowd":0},{"id":3,"image_id":2,"category_id":2,"bbox":[216,73,102.5,427.5],"area":43818.75,"segmentation":[],"iscrowd":0},{"id":4,"image_id":3,"category_id":1,"bbox":[0,0,626,561],"area":351186,"segmentation":[],"iscrowd":0},{"id":5,"image_id":4,"category_id":1,"bbox":[29,208,575.5,271],"area":155960.5,"segmentation":[],"iscrowd":0},{"id":6,"image_id":5,"category_id":1,"bbox":[108,96,378,519],"area":196182,"segmentation":[],"iscrowd":0},{"id":7,"image_id":6,"category_id":1,"bbox":[12,137,609.5,401.5],"area":244714.25,"segmentation":[],"iscrowd":0},{"id":8,"image_id":7,"category_id":2,"bbox":[74,112,484,345.5],"area":167222,"segmentation":[],"iscrowd":0},{"id":9,"image_id":8,"category_id":2,"bbox":[40,150,534.5,416],"area":222352,"segmentation":[],"iscrowd":0},{"id":10,"image_id":9,"category_id":1,"bbox":[46,207,533,262.5],"area":139912.5,"segmentation":[],"iscrowd":0},{"id":11,"image_id":10,"category_id":2,"bbox":[174,290,435,128],"area":55680,"segmentation":[],"iscrowd":0},{"id":12,"image_id":11,"category_id":1,"bbox":[233,97,192,248.5],"area":47712,"segmentation":[],"iscrowd":0},{"id":13,"image_id":11,"category_id":1,"bbox":[221,430,142.5,145],"area":20662.5,"segmentation":[],"iscrowd":0},{"id":14,"image_id":12,"category_id":2,"bbox":[64,348,458,204.5],"area":93661,"segmentation":[],"iscrowd":0},{"id":15,"image_id":13,"category_id":1,"bbox":[161,73,322.5,552],"area":178020,"segmentation":[],"iscrowd":0},{"id":16,"image_id":14,"category_id":2,"bbox":[103,27,422,528],"area":222816,"segmentation":[],"iscrowd":0}]}

[SkalskiP]

I got this error when I tried to open the notebook.

[arami1]

Sorry for the inconvenience.
I downloaded the dataset after doing annotations from roboflow and then uploaded the zip file to google drive. I have used the same code with my own dataset of just 2 classes. Why did this error occur? i am sharing my annotation file please take a look and guide me to remove this error
Detections.getitem not supported for index of type <class 'numpy.ndarray'>
My test annotation file is shown below:
{"info":{"year":"2023","version":"1","description":"Exported from roboflow.ai","contributor":"","url":"[https://public.roboflow.ai/object-detection/undefined","date_created":"2023-03-13T13:21:50+00:00"},"licenses":{"id":1,"url":"https://creativecommons.org/licenses/by/4.0/","name":"CC BY 4.0"}],"categories":[{"id":0,"name":"labels","supercategory":"none"},{"id":1,"name":"rice leaf caterpillar","supercategory":"labels"},{"id":2,"name":"rice leaf roller","supercategory":"labels"}],"images":[{"id":0,"license":1,"file_name":"IP001000460_jpg.rf.7a6253055fc0141a270d2c2275d20f9b.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":1,"license":1,"file_name":"IP000000279_jpg.rf.16cb9316608f637b9ab411191a627c4a.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":2,"license":1,"file_name":"IP000000061_jpg.rf.1fd888279be601c1b8c0e703d2a227bd.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":3,"license":1,"file_name":"IP001000375_jpg.rf.3092d271809e96e0c409a00b98bb04ab.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":4,"license":1,"file_name":"IP001000016_jpg.rf.54b10e8843b41abbb7274d9fec3b2fbe.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":5,"license":1,"file_name":"IP001000179_jpg.rf.1897a09345d8cdab3c8e657bf2656fb0.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":6,"license":1,"file_name":"IP001000455_jpg.rf.005f5066752d3d20d153d83c1c8737a8.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":7,"license":1,"file_name":"IP000000075_jpg.rf.d139beb487b65d39bf1516d8ab548660.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":8,"license":1,"file_name":"IP000000320_jpg.rf.c755fca43e8eef8ff9df0bdb816c376d.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":9,"license":1,"file_name":"IP001000419_jpg.rf.61bd6c3cf3c3a704339ebeea92136de2.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":10,"license":1,"file_name":"IP000000378_jpg.rf.a2d06efb45af1b0fd7d3dc56c8d34e20.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":11,"license":1,"file_name":"IP001000390_jpg.rf.dfe752a3d5e42ae05ae87c4c5a7b0f0a.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":12,"license":1,"file_name":"IP000000134_jpg.rf.df0dd9906e7a8190b572c1ffc6d9c7fc.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":13,"license":1,"file_name":"IP001000070_jpg.rf.51a596f95247ded8d76830faf6cbcef6.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"},{"id":14,"license":1,"file_name":"IP000000212_jpg.rf.f2490939f1b149abe8c8f617c48056a9.jpg","height":640,"width":640,"date_captured":"2023-03-13T13:21:50+00:00"}],"annotations":[{"id":0,"image_id":0,"category_id":1,"bbox":[76,14,215.5,461],"area":99345.5,"segmentation":[],"iscrowd":0},{"id":1,"image_id":0,"category_id":1,"bbox":[344,0,269.5,469.5],"area":126530.25,"segmentation":[],"iscrowd":0},{"id":2,"image_id":1,"category_id":2,"bbox":[63,126,526.5,467],"area":245875.5,"segmentation":[],"iscrowd":0},{"id":3,"image_id":2,"category_id":2,"bbox":[216,73,102.5,427.5],"area":43818.75,"segmentation":[],"iscrowd":0},{"id":4,"image_id":3,"category_id":1,"bbox":[0,0,626,561],"area":351186,"segmentation":[],"iscrowd":0},{"id":5,"image_id":4,"category_id":1,"bbox":[29,208,575.5,271],"area":155960.5,"segmentation":[],"iscrowd":0},{"id":6,"image_id":5,"category_id":1,"bbox":[108,96,378,519],"area":196182,"segmentation":[],"iscrowd":0},{"id":7,"image_id":6,"category_id":1,"bbox":[12,137,609.5,401.5],"area":244714.25,"segmentation":[],"iscrowd":0},{"id":8,"image_id":7,"category_id":2,"bbox":[74,112,484,345.5],"area":167222,"segmentation":[],"iscrowd":0},{"id":9,"image_id":8,"category_id":2,"bbox":[40,150,534.5,416],"area":222352,"segmentation":[],"iscrowd":0},{"id":10,"image_id":9,"category_id":1,"bbox":[46,207,533,262.5],"area":139912.5,"segmentation":[],"iscrowd":0},{"id":11,"image_id":10,"category_id":2,"bbox":[174,290,435,128],"area":55680,"segmentation":[],"iscrowd":0},{"id":12,"image_id":11,"category_id":1,"bbox":[233,97,192,248.5],"area":47712,"segmentation":[],"iscrowd":0},{"id":13,"image_id":11,"category_id":1,"bbox":[221,430,142.5,145],"area":20662.5,"segmentation":[],"iscrowd":0},{"id":14,"image_id":12,"category_id":2,"bbox":[64,348,458,204.5],"area":93661,"segmentation":[],"iscrowd":0},{"id":15,"image_id":13,"category_id":1,"bbox":[161,73,322.5,552],"area":178020,"segmentation":[],"iscrowd":0},{"id":16,"image_id":14,"category_id":2,"bbox":[103,27,422,528],"area":222816,"segmentation":[],"iscrowd":0}]}

[SkalskiP]

@arami1 could you give me a link to your Roboflow project?

[arami1]

https://colab.research.google.com/drive/1nFgpu2bTKLDuFfxeSW6yzzHzIkKtQi9w?usp=sharing

https://app.roboflow.com/datasets-qkode/new_test-zhdrx/browse?queryText=&pageSize=50&startingIndex=0&browseQuery=true

[SkalskiP]

Hi, @arami1! 👋🏻 Thanks a lot for your patience. I spent a bit of time debugging. And I know what is going on.

Quick fix

Remove .with_nms(threshold=0.5) from detections = sv.Detections.from_transformers(transformers_results=results).with_nms(threshold=0.5) line.

Explanation

2 training epochs are not enough for the model to learn detecting objects from your dataset. As a result, your model is not detecting any object on the image. with_nms was not ready to process empty sv.Detections and is raising an exception.

[SkalskiP]

I just merged the fix into the main branch. I will release those changes tomorrow.

[arami1]

ok thank you

[arami1]

I have tried training for 50 epochs but got the same error

[SkalskiP]

This might be because this training does not utilize dataset augmentations, so the model learns quite slowly.

As for the bug, it should be eliminated with the release. It should be done in the next hour or so.

[SkalskiP]

The fix was just released under 0.3.1 tag - https://pypi.org/project/supervision/

I'm closing the issue. Feel free to reopen if you'd still experience any issues.

[arami1]

but I don't want to use data augmentation techniques.
I found this:
You should check whether your problem is like this:
your custom dataset label -> '0' (string type)
your custom DETR predict label -> 0 (int type)
but I don't know how to change my work according to this. can you please help me to solve this issue?

[SkalskiP]

Your problem is not related to the dataset format.

but I don't want to use data augmentation techniques.
Why?

[arami1]

ok, thank you for your guidance. means a lot

[SkalskiP]

No worries ;) Feel free to ask more questions