MobileSAM-in-the-Browser

Description / Rationale

This is an example implementation of MobileSAM in the browser.

Notes

• ONNX runtime web ver. 1.14.0 was used because it was the most compatible with MobileSAM. Unfortunately latest versions on ONNX runtime web did not show good results in segmentation tasks.
• When converting SAM to encoder and decoder some options need to be played with in order to achieve the best results.
• The example uses converted MobileSAM encoder in onnx format. It can be downloaded from here. There are also normal and quantized versions of the decoder, which can be found in "models" folder.

Tech Stack

The project was made possible thanks to SAM, MobileSAM and SAMExporter repositories. In particular, SAM's original model was used in conversion to ONNX and creating a decoder. MobileSAM's original file was used in converting to ONNX and creating encoder. SAMExporter was used in doing conversions to ONNX. Original code was adapted from Tarashish Mishra's SAM in Browser repository.

Demo

See demo here: Demo