Performance and Segmentation Accuracy Issues with SegFormer CoreML Model

[rouzbeh-abadi]

Hello,

I'm integrating the SegFormer CoreML model into an iOS project for real-time semantic segmentation but I'm facing two major issues:

Performance: The model is slower than expected in real-time settings, which is considerably affecting its usability.

Segmentation Accuracy: The model's outputs seem incorrect in terms of class predictions and their spatial locations. The segmentations do not align with the actual positions of the objects in the input image.

Additionally, while initializing the model, I receive a warning in Xcode: "Sort/TopK layer does not support input tensor's shape along vector dimension > 8." I'm wondering if this could be contributing to the issues with the segmentation output.

Any guidance on addressing these issues would be highly appreciated.

Thank you in advance for your assistance.

[john-rocky]

Make sure the input image is cropped correctly.
If you are using VNCoreMLRequest, make sure imageCropAndScaleOption is set to scallFill.