Repository navigation
Releases
sam3-mlx 0.4.0
Breaking changes
run returns a medmlx_core.outputs.RunResult with renamed output roles. outputs["masks"] is now outputs["mask"] and outputs["prediction"] is now outputs["detections"]; the files (masks.npy, prediction.json) are unchanged. The result also holds artifacts, which records that both files are in pixel coordinates of the input image or frame.
The prompt arguments are renamed. The coordinate space is now part of the box and point names.
-from sam3_mlx.medmlx import run
-result = run(image_path="image.png", text_prompt="helmet", box_prompt=[0.5, 0.5, 0.3, 0.3], output_dir="out")
-print(result["outputs"]["masks"])
+from sam3_mlx import run
+result = run(image_path="image.png", prompt="helmet", box_cxcywh_normalized=[0.5, 0.5, 0.3, 0.3], output_dir="out")
+print(result["outputs"]["mask"])point_prompt is now point_xy_normalized. The same renames apply to the arguments file of medmlx run sam3, where text_prompt becomes prompt. from sam3_mlx import run is new; sam3_mlx.medmlx.run still works.
Requires medmlx-core>=0.3,<0.4.
Fixes
Non-finite model predictions now raise ValueError. Before, NaN scores fell below the confidence threshold and a failed inference looked like an image with no detections.
sam3-mlx 0.3.0
Breaking changes
An FP32 or BF16 model now refuses to build in a process that runs TF32. In 0.2.4, starting Python with MLX_ENABLE_TF32=1 silently gave a default-precision model TF32 matrix products. Ask for TF32 by name instead:
- model = build_sam3_image_model() # under MLX_ENABLE_TF32=1
+ model = build_sam3_image_model(precision="tf32")Float images in [0, 1] are converted to 8-bit as upstream SAM 3 does, so their results change. 0.2.4 scaled by 255 and truncated, so about half of arbitrary float pixels came out one level darker; exact multiples of 1/255 and 8-bit images are unchanged.
New
precision="tf32" runs float32 matrix products on the M5 GPU's neural accelerators. It is accepted by build_sam3_image_model, build_sam3_predictor and medmlx run, next to "fp32" (default) and "bf16". On an M5 Max, one image with one text prompt took 0.48-0.53 of the FP32 time with TF32 and 0.74-0.76 with BF16; every level found the same detections as upstream on the six medical test images. Per-image accuracy and speed are in the technical notes.
predict_inst works with the published weights. With enable_inst_interactivity=True, the builder downloads the SAM 1-style interactive head (17 MB) from MedMLX/sam3-interactive.
Fixes
medmlx run no longer refuses 16 GB Macs. sam3 declared a 12 GB peak; it now declares the measured 5.4 GB for one image.
Faster
Peak memory for one image fell from 8.6 to 5.7 GB, and each further text prompt on the same image takes 27% less time (Apple M5 Max, 1008 px). Masks are also upsampled in bounded groups, set by the new Sam3Processor option mask_batch_bytes (64 MiB by default).
sam3-mlx 0.2.4
Supports medmlx-core 0.2.6, required by medmlx 0.4.2.