Skip to content

v3.4.7 - SAM3 ConvRot INT8 Nodes

Choose a tag to compare

View all tags
@ussoewwin ussoewwin released this 29 Aug 20:22
· 8 commits to main since this release
v3.4.7
f781d76
EN 中文

HSWQ v3.4.7 — SAM3 ConvRot INT8 Nodes

This release adds HSWQ SAM3 Loader (ConvRot INT8) and HSWQ SAM3 Detect, letting ComfyUI load and run ConvRot / TensorWise INT8-quantized SAM3 (Segment Anything 3) checkpoints with weights kept in true 8-bit precision in VRAM.

✨ Added

  • HSWQ SAM3 Loader (ConvRot INT8) (HSWQSAM3Loader / HSWQLoadConvRotINT8SAM3, category loaders)
    • Loads SAM3 checkpoints carrying int8_tensorwise comfy_quant metadata via MixedPrecisionOps, keeping Linear layers as QuantizedTensor (TensorWiseINT8Layout) — true INT8 in VRAM (~525 MB for SAM3.1 Multiplex, about half of FP16)
    • Searches diffusion_models / sams / detection / checkpoints folders
    • Non-INT8 (FP16 etc.) checkpoints load normally with stock settings
  • HSWQ SAM3 Detect (HSWQSAM3Detect, category HSWQ/Detection)
    • Open-vocabulary detection & segmentation with text (CONDITIONING), box (BBOXES), and point prompts (JSON pixel coords)
    • Outputs masks (MASK), bboxes (BBOXES), and pass-through image (IMAGE)
    • Options: threshold (0.50), refine_iterations (SAM decoder refinement, default 2), individual_masks
  • comfy_kitchen INT8 GEMM safety fallback (_patch_comfy_kitchen_int8_gemm_fallback)
    • Non-multiple-of-4 dimensions (e.g. boxRPB_embed_x K=2) automatically fall back to float precision instead of crashing cuBLAS INT8 GEMM

🐛 Fixed

  • SAM3 INT8 runtime crash under DynamicVRAM (aimdo) — ValueError: Buffer too small: needs 6291456 bytes, but only has 3164160. in resolve_cast_module_with_vbar (6d4f3f8)
    • Root cause: the vbar buffer is allocated with the INT8 payload size (int8 data + scale); after the runtime weight guard replaces weights with FP16, the float16-sized cast geometry no longer fits
    • Fix: _strip_dynamic_vram_attrs drops the vbar state (_v / _prefetch / _v_signature / _v_block, after vbar_unpin) so cast_bias_weight falls back to the regular cast path
  • SAM3 CLIP loading — "clip missing" → noisy salt-and-pepper masks (8c20913)
    • Root cause: sd-level and _clip_stash in_proj_weight pre-splits broke ComfyUI's transformers_convert remap (it expects the fused in_proj_weight form); INT8 checkpoints store language_backbone already split into q/k/v, which transformers_convert cannot remap
    • Fix: removed both pre-splits; process_clip_state_dict now remaps leftover encoder.* keys to sam3_clip.transformer.text_model.encoder.layers.N.self_attn.q_proj
    • Verified: fp16 and INT8 both score ~0.98 on a real image + "person"; masks are clean uniform white (previously ~0.27 / NaN with 1000+ speckle components)

📚 Documentation

  • HSWQ SAM3 ConvRot INT8 — Complete Technical Guide — overview, files created/modified, full code, per-function explanations (baseline d33862a)
  • README / Chinese README updated: node usage, example workflow, FP16 compatibility
  • CHANGELOG v3.4.7 (EN + 中文)

✅ Compatibility

  • FP16 SAM3 checkpoints fully supported — HSWQ SAM3 Loader falls back to stock loading (no MixedPrecisionOps); HSWQ SAM3 Detect produces equivalent masks on both paths
  • Works with stock CheckpointLoaderSimple thanks to the CLIP remap patch (no "clip missing")
  • Compatible with ComfyUI native SAM3 nodes and HSWQ SAM3 Detect
Assets 2
Loading