For anyone struggling with v3.1

[heyoeyo]

Hey everyone!

There seems to be a variety of issues (#550, #544, #542, #526, #506, etc.) with the v3.1 update, mostly around device usage (cpu vs. gpu), memory usage or even just loading the model.

For anyone having issues with the v3.1 model, I have a repo (MuggledSAM) that can run it (along with v1, v2 and v3.0 models) with support for cpu use, float16 on GPU and without any of the memory buildup problems. It might run on Mac as well, though it's untested. If you already installed SAM3, you can probably re-use the same env.

It also supports running at reduced resolutions, which works especially well with the v3.1 multiplexing features. For example, here's a screen capture (from running this script with the SAM3 dance video) running at a reduced resolution of 336px, which manages to track 4 objects at 50fps on a 3090:

sam3v1_mplex_336px.mp4

The model API is different from the original SAM implementation but there's a folder of example scripts for how to use the model for different tasks.

[dvdsosa]

Wow, your repo is just incredible! Thank you so much for your work, it saved me a lot of hours 🙌🏼

[heyoeyo]

Thanks for the kind words!