Segment Anything from META: Network implementation. Has someone started already?

[vgonzalezd]

Is your feature request related to a problem? Please describe.
Is it someone implementing in MONAI the segment anything model in MONAI?
https://github.com/facebookresearch/segment-anything
My concern is the weight matching. We can help with it if someone starts already.

Describe the solution you'd like
This will make more students download and use MONAI
For sure it will be an useful network for MONAILABEL

Describe alternatives you've considered
N/A

Additional context
Add any other context or screenshots about the feature request here.

[yashika-git]

@wyli, @ericspod,
Is the implementation of Segment Anything still required?
If yes, can I try working on this?

[KumoLiu]

Hi @yashika-git, yes, welcome to your contribution.
Please refer to the CONTRIBUTION GUIDE as well.

[yashika-git]

Thanks, I've started to look how it can be implemented.

[yashika-git]

Could someone please provide a little more details about the expected outcome of the implementation?
Should the users be able to infer and/or fine-tune the SAM model on custom datasets?
Thank you

[KumoLiu]

Hi @yashika-git, yes first I think we should enable users to load the existing pre-trained weights, also I think they could infer and fine-tune the model on their custom datasets.

[yashika-git]

Hi @KumoLiu, I have a very naive question: What exactly needs to be done to integrate SAM with MONAI? Does it involve modifying SAM's codebase to utilize MONAI's existing code, such as MONAI's networks and layers?
I apologize for the naive question; I'm a beginner in open source and am eager to learn through contributions.

[KumoLiu]

Hi @yashika-git, For existing parts that can be replaced by the MONAI, you should use the components of MONAI as much as possible.