Skip to content

sam3-mlx 0.3.0

Choose a tag to compare

@sandovaljoseph sandovaljoseph released this 10 Nov 09:37
· 10 commits to main since this release

Breaking changes

An FP32 or BF16 model now refuses to build in a process that runs TF32. In 0.2.4, starting Python with MLX_ENABLE_TF32=1 silently gave a default-precision model TF32 matrix products. Ask for TF32 by name instead:

- model = build_sam3_image_model()                  # under MLX_ENABLE_TF32=1
+ model = build_sam3_image_model(precision="tf32")

Float images in [0, 1] are converted to 8-bit as upstream SAM 3 does, so their results change. 0.2.4 scaled by 255 and truncated, so about half of arbitrary float pixels came out one level darker; exact multiples of 1/255 and 8-bit images are unchanged.

New

precision="tf32" runs float32 matrix products on the M5 GPU's neural accelerators. It is accepted by build_sam3_image_model, build_sam3_predictor and medmlx run, next to "fp32" (default) and "bf16". On an M5 Max, one image with one text prompt took 0.48-0.53 of the FP32 time with TF32 and 0.74-0.76 with BF16; every level found the same detections as upstream on the six medical test images. Per-image accuracy and speed are in the technical notes.

predict_inst works with the published weights. With enable_inst_interactivity=True, the builder downloads the SAM 1-style interactive head (17 MB) from MedMLX/sam3-interactive.

Fixes

medmlx run no longer refuses 16 GB Macs. sam3 declared a 12 GB peak; it now declares the measured 5.4 GB for one image.

Faster

Peak memory for one image fell from 8.6 to 5.7 GB, and each further text prompt on the same image takes 27% less time (Apple M5 Max, 1008 px). Masks are also upsampled in bounded groups, set by the new Sam3Processor option mask_batch_bytes (64 MiB by default).