[Question] How to add semantic_label using InteractiveSceneCfg

[yizhengzhang1]

Hi,

I want to know if there are some methods to add semantic_label when creating prim using InteractiveSceneCfg to generate a scene.

Or this is not done yet in orbit framework.

[yizhengzhang1]

If I load an usd file which contains lots of objects, how to label these objects one by one automaticly

[yizhengzhang1]

like the function apply semantic data using prim names in https://docs.omniverse.nvidia.com/isaacsim/latest/replicator_tutorials/tutorial_replicator_getting_started.html

[Mayankm96]

We support semantic tags on the "entire" imported prim. You can set that from here: https://isaac-orbit.github.io/orbit/source/api/orbit/omni.isaac.orbit.sim.spawners.html#omni.isaac.orbit.sim.spawners.SpawnerCfg.semantic_tags

For more advanced usage, I think it is better if you pre-process your USD file and have the semantic labels specified on individual prims in that USD file. Alternately, you can write your own spawn function where you apply these labels at runtime.

[yizhengzhang1]

Thanks, semantic tags should solve the problem.

Is it possiable to run the function from "Semantics Schema Editor" by python code in orbit?
The GUI has an "add" button which can add all the tag using prim names.
It should be convenient if it can use python code directly.

[yizhengzhang1]

I'm not sure if the semantic tags work well.

I try the code "source/standalone/tutorials/04_sensors/run_usd_camera.py", adding the "semantic_segmentation" option to camera and save the images.

camera_cfg = CameraCfg(
        prim_path="/World/Origin_.*/CameraSensor",
        update_period=0,
        height=480,
        width=640,
        data_types=["rgb", "distance_to_image_plane", "normals", "semantic_segmentation"],
        spawn=sim_utils.PinholeCameraCfg(
            focal_length=24.0, focus_distance=400.0, horizontal_aperture=20.955, clipping_range=(0.1, 1.0e5)
        ),
    )

The code contains:

_ = prim_utils.create_prim(
            f"/World/Objects/Obj_{i:02d}",
            prim_type,
            translation=position,
            scale=(0.25, 0.25, 0.25),
            semantic_label=prim_type,
        )

which already set the semantic_label.

However, the value of semantic_segmentation image is 1, which represents the "UNLABELLED".

Is there some bugs here?

[Mayankm96]

For me this looks fine. The labels are:

{
    "(0, 0, 0, 0)": {
        "class": "BACKGROUND"
    },
    "(0, 0, 0, 255)": {
        "class": "UNLABELLED"
    },
    "(140, 255, 25, 255)": {
        "class": "cube"
    },
    "(140, 25, 255, 255)": {
        "class": "cylinder"
    },
    "(255, 197, 25, 255)": {
        "class": "cone"
    }
}

[yizhengzhang1]

Just update the latest code.
Works well now, thanks!
I believe this commit fix this bug: 8dea21a

[yizhengzhang1]

Although the semantic camera is correct, the semantic tag is still wrong.
Its semantic label is the path in the simulation, not the value set when generating