fix(sam3): stop IndexError on single-point visual prompts in serialize_prompt

[bigbitbus]

Summary

SAM3 interactive Smart Select returns a 500 (IndexError: list index out of range) whenever a visual prompt contains exactly one point — e.g. a single smart-select click, or converting a tiny (<5px) box to a polygon. This surfaces in the app as "Smart Select didn't respond. Try again in a moment." because the deterministic 500 exhausts the client's 3 retries.

The crash is in serialize_prompt() (the mask-input cache-key builder), not in the model.

Root cause

segment_with_visual_prompts() passes per-image point coordinates shaped (num_prompts, num_points, 2) to serialize_prompt(). But serialize_prompt iterated only the leading (prompt) axis, treating each prompt's entire point list as if it were one [x, y] pair:

for coord, label in zip(coords_list, labels_list):   # coords_list is (P, N, 2)
    "y": coord[1] if isinstance(coord, (list, tuple)) else 0,   # <- IndexError

For a single point the array is (1, 1, 2), so the first coord is [[x, y]] — a length-1 list — and coord[1] raises IndexError. With 2+ points it doesn't crash but silently hashes garbage (x/y become lists), so the cache key was wrong all along. Only the non-preview path hits it (serialize_prompt runs only when mask-input caching is enabled), matching the symptom.

The older inference/models/sam3/visual_segmentation.py never had this — it has no serialize_prompt step and feeds (P, N, 2) straight into predict_inst. The bug came in with the pytorch rewrite's low-res-logits mask cache.

Fix

Iterate prompts then points, matching the SAM2 serialize_prompt reference and _predict_for_single_image, which both consume (P, N, 2) correctly. Also fixes the silently-wrong cache keys for multi-point prompts.

Tests

Added unit tests in test_sam3_client_hashes.py (test_serialize_prompt_single_point_does_not_raise, test_serialize_prompt_multiple_points_flattens_in_order). Verified the pre-fix loop raises IndexError on (1,1,2) and the fix produces the expected output. Validated on ck8s-stg via a source-overlay test image — single-point Smart Select now returns POST /sam3/visual_segment 200.

Release (included in this PR)

inference-models is consumed by the server image as a pinned PyPI wheel, so this fix ships only once a new wheel is published. This PR therefore also:

• bumps inference_models/pyproject.toml + uv.lock to 0.29.1 (the publish workflow uses skip-existing: true, so a bump is required to upload), and
• raises the pins in requirements.{gpu,cpu,vino,jetson}.txt to ~=0.29.1 so the server image cannot resolve the unfixed 0.29.0.

Post-merge: run the "Publish Inference Models Wheels to PyPi" workflow (or cut a release) to publish 0.29.1, then rebuild/redeploy the inference server image.

Notes / follow-up (not in this PR)

InferenceModelsSAM3InteractiveAdapter.segment_image dropped the legacy empty-prompt fallback (if not any(args.values()): ...) that both the old path and _predict_for_single_image still have. Not the trigger here, but worth restoring for parity.

🤖 Generated with Claude Code