OIDv5 - Segmentation Masks Support

[Auth0rM0rgan]

Hey @monocongo, Thanks for sharing your code!

I would like to use OIDv5 segmentation masks to train my instance segmentation approach with OIDv5.
It would be nice to have:

• Toolkits to download OIDv5 segmentation masks.
• Convert OIDv5 segmentation annotation files to COCO Json format.

Thanks!

[monocongo]

Thanks @Auth0rM0rgan

If you can please post a link to documentation that describes the COCO JSON format for segmentation, so I can use that as a guide. I have used COCO in the past and may have some old code to use as a guide, but it was for bounding boxes and not segmentation mask polygons.

Related to cvdata issue #145

[monocongo]

Create COCO Annotations from Scratch

[woutgg]

Hi, thanks for sharing this project!

As I needed segmentation masks with the images, I forked your repo and continued the work you had already done. Apparently there is no file-level hosting of the mask files, so the script downloads all the segmentation zip files (about 4.4GB) and extracts from them what is actually needed.

I've also added some more progress bars and log messages since (with my internet connection) the initial wait time was quite long and I wanted to have a bit more progress info available.

If you are interested, please have a look at my fork. I'd be happy to open a PR and rework things where necessary.

[woutgg]

BTW, I realize this issue has been closed, but I was under the impression that this might be a good place to continue on the matter anyway. Let me know if you disagree.

[monocongo]

@woutgg this is very welcome news, thanks for your work on this!

BTW I have noticed that there's now an OID version 6. Do you know what the differences are in terms of how this new version may affect this code? For example, have any download URLs changed, etc.?

If you can further summarize what has changed and/or been added in your fork then please update here in the comments (I'm lazy!).

Thanks again for your efforts, quite helpful.

[woutgg]

BTW I have noticed that there's now an OID version 6. Do you know what the differences are in terms of how this new version may affect this code? For example, have any download URLs changed, etc.?

I was also wondering this and I do not think it affects the code at all. In their introductory text here, this is the main difference: "In V6 we added 507k localized narratives: multimodal descriptions of images consisting of synchronized voice, text, and mouse traces over the objects being described."

The class names, most bbox CSVs as well as the segmentation files still point to V5; the exception to this is the train bbox CSV but I didn't check the difference yet.

If you can further summarize what has changed and/or been added in your fork then please update here in the comments (I'm lazy!).

Sure :), here goes:

• Commit 1 fixes some small errors in the preliminary changes you made.
• Commit 2 refactors the HTTP downloads into a function that runs a progress bar along (mainly for the large training bbox+segmentation CSVs) and adds a few 'heartbeat' log messages.
• Commit 3 adds the mask downloading. This happens by first fetching the zip files for each split section (16, as they are binned to the first hex char), then each masking image as listed in the CSV is extracted from the appropriate archive and moved into place. The zip files are opened only once per class label iteration, to get decent performance - this could be improved to opening them only once per split section.

I still have to double check if the resultant file output is acutally correct, so beware of bugs.

[woutgg]

So apparently the output was not correct, I had overlooked that your code filters on the bbox attributes (IsOccluded etc.).
I've added a commit that fixes this, and another that skips downloading of existing files.

This time I checked the file counts created by the script against manual counts (well with some help from awk and friends) and they match up.

[monocongo]

Good catch. We should probably add options for keeping or filtering bboxes with the IsOccluded, IsGroupOf, etc. flags.

[woutgg]

Indeed, my thought as well. It's even on my todo-list already, but since I don't need it at the moment I'm leaving it for now.

If you want me to turn the current changes into a PR, just let me know.

[monocongo]

Yes, absolutely, you've made some good contributions here that I'd like to be folded into the main repository. Thanks!

[woutgg]

There you go!

I'm currently trying out a model implementation that uses OID annotations, but in case that fails I might implement the COCO conversion as well.

[Rm1n90]

Hey @woutgg, @monocongo,

I would like to ask you that do you have any plan to support to download OID segmentation mask and segmentation annotations in Coco format?

[monocongo]

I currently don't have the bandwidth to devote to this conversion to the COCO format. Surprisingly there doesn't seem to be such a converter available as an open source tool, but I have not done an exhaustive search. I have started work on this over in my computer vision datasets manipulation package repository, cvdata (see this issue), and that seems to be a more logical place for a conversion such as this, but no immediate plans to complete that soon. PRs are very welcome!

[nicolasmetallo]

Thanks for the excellent work on this library! I did a converter some time ago from OID to PASCAL VOC here: https://github.com/nicolasmetallo/train_custom_detection_with_OID maybe some of it it's useful. There are many available converters from PASCAL to COCO.