Skip to content

feat(sam3): add class_mapping support to SAM3 v3 block#2196

Open
felipe-tomino wants to merge 2 commits intomainfrom
tomino/sam3-class-mapping
Open

feat(sam3): add class_mapping support to SAM3 v3 block#2196
felipe-tomino wants to merge 2 commits intomainfrom
tomino/sam3-class-mapping

Conversation

@felipe-tomino
Copy link
Copy Markdown

Summary

Adds an optional class_mapping parameter to the roboflow_core/sam3@v3 workflow block that remaps prediction class names before returning results.

Use case: Rapid's non-English users type class names like "gato" (Spanish for cat). SAM3 works best with English prompts, so Rapid sends "cat" as the prompt but needs the API to return predictions with class: "gato" — what the user expects.

How it works:

  • New optional field: class_mapping: Dict[str, str] (e.g., {"cat": "gato"})
  • After inference completes (all execution modes: remote, local, SDK), prediction class_name values are remapped
  • If no mapping exists for a class, it passes through unchanged (backwards compatible)
  • Field is optional with None default — no behavior change for existing workflows

Changes

  • inference/core/workflows/core_steps/models/foundation/segment_anything3/v3.py

    • Added class_mapping field to BlockManifest
    • Added class_mapping parameter to run() method
    • Added _apply_class_mapping() static method that remaps class_name in sv.Detections data
    • Applied after all 3 execution paths (remote, local, SDK)
  • tests/workflows/unit_tests/core_steps/models/foundation/test_segment_anything3_v3.py (new)

    • 6 test cases: full mapping, partial mapping, no-match, multiple images, empty result, empty mapping

Test plan

  • Unit tests pass: pytest tests/workflows/unit_tests/core_steps/models/foundation/test_segment_anything3_v3.py

@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Apr 2, 2026

CLA assistant check
All committers have signed the CLA.

Add an optional `class_mapping` parameter to the SAM3 v3 workflow block
that remaps prediction class names before returning results.

This enables Rapid to send English prompts to SAM3 for optimal detection
while returning the user's original class names (e.g., non-English terms)
in the API response. For example, SAM3 detects with prompt "cat" but
returns predictions with class "gato" when class_mapping={"cat": "gato"}.
@felipe-tomino felipe-tomino force-pushed the tomino/sam3-class-mapping branch from 18b8cb0 to 6719d7c Compare April 2, 2026 17:20
Copy link
Copy Markdown
Contributor

@dkosowski87 dkosowski87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add also a test for the whole workflow block with this scenario pls.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants