Support for Universal Segmentation Models:Mask2Former, MaskFormer and OneFormer

[srikrishnavignesh]

HuggingFace Universal Segmentation Model Support (MaskFormer, Mask2Former, OneFormer)

Motivation

I was working on the Solafune aerial tree canopy segmentation competition using Mask2Former and needed SAHI's sliced inference to improve detection of small regions. Since SAHI didn't have support for these models, I built a prototype integration. It worked well enough to help me achieve a rank of 34 in the competition, so I thought it might be useful to contribute back to the community.

What the prototype supports

• Models: Mask2FormerForUniversalSegmentation, MaskFormerForInstanceSegmentation, OneFormerForUniversalSegmentation
• Segmentation types: instance, semantic, panoptic
• Handles post-processing differences between MaskFormer/Mask2Former and OneFormer processor APIs
• Converts all output formats into SAHI's expected binary mask + segments_info format
• min_mask_area parameter to filter small/noisy mask fragments

Current state

I have a working prototype tested across all three models and segmentation types. I can raise a draft PR if you'd like to take a look at the code. If the approach looks good, I'm happy to add tests and docstrings before a formal PR.

This would also be my first open source contribution, so any feedback on the approach or how you'd like it structured is very welcome!

Let me know what you think!

[onuralpszr]

@srikrishnavignesh we have quite big refactor happen in latest changes so if you up for challenge, I can check your PR and happy see contribution go for it.

[srikrishnavignesh]

I have made a draft PR onuralpszr. Please review it. If thats okay , I can add the relevant tests as well.

[onuralpszr]

PR has been merged @srikrishnavignesh thank you :)