How would you segment everything in an image and output masks

[kuaiqushangzixiba]

I don't want to manually set points or boxes, I want to directly segment all the objects in the graph

[chongzhou96]

Since EdgeSAM follows the same encoder-decoder architecture as SAM, the everything mode will infer the decoder 32x32=1024 times, which is very inefficient. So I don't recommend doing that.

But if you really want to try it out, here is the script:

from segment_anything import sam_model_registry, SamAutomaticMaskGenerator

sam = sam_model_registry["edge_sam"](checkpoint="weights/edge_sam_3x.pth")
sam = sam.to(device=device)
sam.eval()

mask_generator = SamAutomaticMaskGenerator(sam)

# TODO: read an image and convert it to numpy.ndarray

masks = mask_generator.generate(image)

[chongzhou96]

Let's say the encoder of the original SAM on a 2080 Ti takes 235 ms and that of the EdgeSAM takes 5 ms and the decoder of both models takes 1 ms. The total execution times in everything mode are 235 + 1024 * 1 = 1259 ms and 5 + 1024 * 1 = 1029 ms for SAM and EdgeSAM respectively.

So, if VRAM is not your bottleneck, I would recommend using the original SAM instead.

[CharlesPikachu]

I don't want to manually set points or boxes, I want to directly segment all the objects in the graph

Hello, sssegmentation also supported EdgeSAM now, you can write the following codes to achieve this goal after installing sssegmentation,

import cv2
import torch
import numpy as np
import matplotlib.pyplot as plt
from ssseg.modules.models.segmentors.sam.visualization import showanns
from ssseg.modules.models.segmentors.edgesam import EdgeSAMAutomaticMaskGenerator

# read image
image = cv2.imread('images/dog.jpg')
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
# mask generator
mask_generator = EdgeSAMAutomaticMaskGenerator(use_default_edgesam=True, device='cuda')
# generate masks on an image
masks = mask_generator.generate(image)
# show all the masks overlayed on the image
plt.figure(figsize=(20, 20))
plt.imshow(image)
showanns(masks)
plt.axis('off')
plt.savefig('mask.png')