ovrtx: expose semantic segmentation idToLabels

[rilei-nvidia]

Description

This PR brings our ovrtx renderer integration's semantic_segmentation annotator to full parity with Isaac RTX's. This involves building the idToLabels dict mapping the semantic ID or color to the corresponding labels - which IsaacRTX supplies through Replicator.

A new ovrtx_annotator_utils.py is introduced encapsulating the logic of decoding ovrtx SemanticIdMap raw array of C++ structs. In the future we can remove this implementation in favor of logic exposed natively in ovrtx or through ovannotators.

While adding the tests to validate the consistency of idToLabels output - I also noticed that the IsaacRTX's pseudo-random colors generated from the semantic IDs was slightly different than ovrtx's. So that's been fixed and golden images regenerated.

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)

Screenshots

See golden images update

Checklist

• I have read and understood the contribution guidelines
• I have run the pre-commit checks with ./isaaclab.sh --format
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• I have updated the changelog and the corresponding version in the extension's config/extension.toml file
• I have added my name to the CONTRIBUTORS.md or my name already exists there

[greptile-apps]

Greptile Summary

This PR brings the OVRTX renderer's semantic_segmentation annotator to parity with Isaac RTX by decoding the SemanticIdMap render var into an idToLabels mapping (keyed by color or raw ID) and exposing it through camera.data.info. It also fixes the color_hash kernel to use 32-bit wraparound arithmetic matching omni.replicator's randomColoursCPU, adds a colorize_semantic_segmentation config flag, and regenerates affected golden images.

• New ovrtx_annotator_utils.py decodes the packed C-struct SemanticIdMap buffer into a Python dict and builds the idToLabels mapping; build_semantic_id_to_labels merges reserved IDs with decoded entries and optionally maps IDs to RGBA color keys via a Warp kernel.
• color_hash fix in ovrtx_renderer_kernels.py corrects the intermediate arithmetic to stay in 32-bit (matching the Replicator reference), which changes colorized segmentation output and required golden image regeneration.
• Tests added cover round-trip decode, corrupt-buffer rejection, and both colorized/non-colorized idToLabels paths, plus contract tests for the new renderer config flag and renderer_info forwarding.

Confidence Score: 4/5

Safe to merge for the feature goal; the three issues flagged are hardening and performance concerns rather than conditions that break existing callers.

The new ovrtx_annotator_utils.py has a label-bounds check that uses the full buffer size instead of excluding the 4-byte trailing count, and a dict merge order that lets decoded entries overwrite the well-known BACKGROUND/UNLABELLED reserved labels if the OVRTX runtime ever includes IDs 0/1 in the SemanticIdMap. Neither condition is triggered by the current runtime, but both are straightforward to fix. The per-frame GPU kernel + sync for color-key computation is unnecessary overhead since semantic IDs are stable across frames. The kernel fix, the new colorize flag, and the idToLabels wiring are all correct.

source/isaaclab_ov/isaaclab_ov/renderers/ovrtx_annotator_utils.py — the bounds check and merge order both warrant a second look before shipping.

Important Files Changed

Filename	Overview
source/isaaclab_ov/isaaclab_ov/renderers/ovrtx_annotator_utils.py	New module decoding SemanticIdMap into idToLabels; contains a loose upper-bound check that permits labels to span into the trailing entry-count bytes, and a merge order that lets decoded entries override reserved IDs 0/1.
source/isaaclab_ov/isaaclab_ov/renderers/ovrtx_renderer.py	Wires up SemanticIdMap decoding into renderer_info and forwards it to camera_data.info; adds colorize_semantic_segmentation config flag; changes semantic seg output spec from hardcoded RGBA to config-driven. Logic is sound; _process_semantic_id_map runs color key computation every frame.
source/isaaclab_ov/isaaclab_ov/renderers/ovrtx_renderer_kernels.py	Fixes color_hash to use 32-bit wraparound arithmetic matching omni.replicator's randomColoursCPU; adds uint32-to-int32 overload for extract_all_tiles_kernel needed by non-colorized semantic output.
source/isaaclab_ov/test/test_ovrtx_annotator_utils.py	New unit tests for the annotator utilities covering round-trip decode, empty buffer, corrupt entry count, and both colorized/non-colorized build paths. Good coverage; colorized test conditionally skips without CUDA.
source/isaaclab_tasks/test/rendering_test_utils.py	Adds maybe_validate_semantic_segmentation helper and wires up idToLabels exact-equality checks for cartpole and shadow_hand tasks with hardcoded color keys.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant R as OVRTXRenderer.render()
    participant IDMap as frame.render_vars[SemanticIdMap]
    participant Utils as ovrtx_annotator_utils
    participant RD as OVRTXRenderData.renderer_info
    participant RO as OVRTXRenderer.read_output()
    participant CD as camera_data.info

    R->>IDMap: "map(device=CPU)"
    IDMap-->>Utils: np.ndarray (raw bytes)
    Utils->>Utils: decode_semantic_id_map()
    Utils->>Utils: build_semantic_id_to_labels()
    Note over Utils: colorize=True → semantic_color_keys()<br/>(Warp kernel + host-device sync)
    Utils-->>RD: "renderer_info[semantic_segmentation] = idToLabels"
    R->>RO: read_output(render_data, camera_data)
    RO->>CD: "camera_data.info[semantic_segmentation] = renderer_info value"

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant R as OVRTXRenderer.render()
    participant IDMap as frame.render_vars[SemanticIdMap]
    participant Utils as ovrtx_annotator_utils
    participant RD as OVRTXRenderData.renderer_info
    participant RO as OVRTXRenderer.read_output()
    participant CD as camera_data.info

    R->>IDMap: "map(device=CPU)"
    IDMap-->>Utils: np.ndarray (raw bytes)
    Utils->>Utils: decode_semantic_id_map()
    Utils->>Utils: build_semantic_id_to_labels()
    Note over Utils: colorize=True → semantic_color_keys()<br/>(Warp kernel + host-device sync)
    Utils-->>RD: "renderer_info[semantic_segmentation] = idToLabels"
    R->>RO: read_output(render_data, camera_data)
    RO->>CD: "camera_data.info[semantic_segmentation] = renderer_info value"

Loading

_{Reviews (1): Last reviewed commit: "ovrtx: expose semantic segmentation idTo..." | Re-trigger Greptile}