[Question] Dynamically create fixed joint between two rigid bodies during simulation runtime

[xdd7561]

Question

Hi all,

I have an object and robot manipulator. During simulation, I want to rigidly attach the end effector of the manipulator to the object to simulate the object being grasped.

I thought I could create a fixed joint between the object and end effector dynamically during runtime. However, I'm running into some trouble doing this.

1. There is an example of a similar situation in Dynamically Attach Two Rigid Bodies at Runtime in IsaacLab #2472, but perhaps the provided code is outdated? I cannot import get_scene_interface.

Traceback (most recent call last):
  File "isaaclab_script.py", line 108, in <module>
    from omni.physx import get_scene_interface
ImportError: cannot import name 'get_scene_interface' from 'omni.physx' (/home/.../conda_env/lib/python3.11/site-packages/isaacsim/extscache/omni.physx-107.3.18+107.3.1.lx64.r.cp311.u353/omni/physx/__init__.py)

2. I also saw that [Question] How to change the initial pose of a static articulation/rigidbody? #4025 suggests using USD API. However, when I attempt to do this, upon creating a fixed joint, the end-effector snaps to the initialization position of the object. I suspect because the USD values don't update during gpu simulation even when the physx values do?

[Warning] [omni.physx.plugin] PhysicsUSD: CreateJoint - found a joint with disjointed body transforms, the simulation will most likely snap objects together: /World/envs/.../EefObjectFixed

Thus, I'm hoping to get some guidance on how correctly create a fixed joint between two rigid bodies during simulation runtime. Any help is appreciated.

Build Info

Describe the versions that you are currently using:

• Isaac Lab Version: 2.2.0
• Isaac Sim Version: 5.0.0

[RandomOakForest]

Thank you for posting this. To rigidly attach the end-effector to the object at runtime in Isaac Sim / Isaac Lab, you may need to (1) stop using the deprecated get_scene_interface path, and (2) create a USD FixedJoint (or PhysX attachment) whose pose matches the current simulated transforms of the two bodies before the next physics step, so PhysX does not “snap” them together. For most grasp-like use cases in Isaac Lab, the easiest “runtime rigid attach” is to avoid low-level joints and instead use the higher-level grasping/attachment utilities that sit on top of PhysX.

[xdd7561]

Thank you for the response. Would you be able to provide a code snippet/example of attaching 2 rigid bodies (doesn't have to be attaching an object to end effector, just 2 objects is fine) during runtime?

[xdd7561]

@RandomOakForest Given two objects (Eg cubes) next to each other, after a few simulation steps, I used PhysX attachment api to connect them but the bodies ended up flying away the step after connecting (only on gpu). On cpu it works fine, but on GPU the objects teleport to original positions before rigidly attaching. I'm not sure if I'm doing it correctly -- could you provide guidance on where I can find how to correctly do this? Appreciate the help!

from isaacsim import SimulationApp
simulation_app = SimulationApp({"headless": False})
import numpy as np
import torch
from isaacsim.core.api import World
from isaacsim.core.api.objects import DynamicCuboid
from isaacsim.core.simulation_manager import SimulationManager
import omni.usd
from omni.physx.scripts import utils as physx_utils


# Toggle CPU vs GPU
USE_GPU = False  # True for GPU PhysX, False for CPU

if USE_GPU:
    backend = "torch"
    device = "cuda:0"
else:
    backend = "numpy"
    device = None  # CPU


def make_vel(x, y, z):
    if USE_GPU:
        return torch.tensor([x, y, z], device=device, dtype=torch.float32)
    else:
        return np.array([x, y, z], dtype=np.float32)


def dist_between(pos_a, pos_b):
    if USE_GPU:
        if not isinstance(pos_a, torch.Tensor):
            pos_a = torch.tensor(pos_a, device=device, dtype=torch.float32)
        if not isinstance(pos_b, torch.Tensor):
            pos_b = torch.tensor(pos_b, device=device, dtype=torch.float32)
        return torch.linalg.norm(pos_a - pos_b).item()
    else:
        return float(np.linalg.norm(pos_a - pos_b))


def weld_rigid_bodies(prim_path_a: str, prim_path_b: str, joint_type: str = "Fixed"):
    stage = omni.usd.get_context().get_stage()

    prim_a = stage.GetPrimAtPath(prim_path_a)
    prim_b = stage.GetPrimAtPath(prim_path_b)

    if not prim_a or not prim_a.IsValid():
        raise RuntimeError(f"weld_rigid_bodies: Prim not found or invalid: {prim_path_a}")
    if not prim_b or not prim_b.IsValid():
        raise RuntimeError(f"weld_rigid_bodies: Prim not found or invalid: {prim_path_b}")

    joint_prim = physx_utils.createJoint(stage, joint_type, prim_a, prim_b)

    print(
        f"weld_rigid_bodies: Created {joint_type} joint at {joint_prim.GetPath()} "
        f"between '{prim_path_a}' and '{prim_path_b}'"
    )

    return joint_prim


# Scene setup
world = World(backend=backend, device=device)
physics_ctx = world.get_physics_context()
if USE_GPU:
    physics_ctx.enable_gpu_dynamics(True)
    physics_ctx.enable_fabric(True)
    print("[config] Using GPU PhysX pipeline (torch/cuda, Fabric ON)")
else:
    physics_ctx.enable_gpu_dynamics(False)
    physics_ctx.enable_fabric(False)
    print("[config] Using CPU PhysX pipeline (numpy, Fabric OFF)")

world.scene.add_default_ground_plane()

big_cube_path = "/World/big_cube"
small_cube_path = "/World/small_cube"

# Big cube: moves toward the small cube
big_scale = np.array([0.4, 0.4, 0.4], dtype=np.float32)
small_scale = np.array([0.2, 0.2, 0.2], dtype=np.float32)

big_cube = world.scene.add(
    DynamicCuboid(
        prim_path=big_cube_path,
        name="big_cube",
        position=np.array([-1.0, 0.0, 0.5], dtype=np.float32),
        scale=big_scale,
        size=1.0,
        color=np.array([0.0, 0.0, 1.0], dtype=np.float32),
    )
)

# Small cube: initially stationary
small_cube = world.scene.add(
    DynamicCuboid(
        prim_path=small_cube_path,
        name="small_cube",
        position=np.array([0.0, 0.0, 0.5], dtype=np.float32),
        scale=small_scale,
        size=1.0,
        # <<< same here
        color=np.array([1.0, 0.0, 0.0], dtype=np.float32),
    )
)

world.reset()

stage = omni.usd.get_context().get_stage()

# Params
half_big = 0.5 * 1.0 * float(big_scale[0])
half_small = 0.5 * 1.0 * float(small_scale[0])
contact_distance = half_big + half_small  # ≈ 0.3
weld_surface_epsilon = 0.1

print(f"Contact distance (center-to-center) ~ {contact_distance:.3f} m")
print(f"Weld when surface gap < {weld_surface_epsilon:.3f} m")

welded = False
weld_frame = -1
impulse_applied = False

num_steps = 1000
target_speed = 1.0
upward_speed = 15.0


# Simulation loop
for frame in range(num_steps):
    if not welded:
        big_cube.set_linear_velocity(make_vel(target_speed, 0.0, 0.0))

    pos_big, _ = big_cube.get_world_pose()
    pos_small, _ = small_cube.get_world_pose()
    distance = dist_between(pos_big, pos_small)
    surface_gap = distance - contact_distance

    if not welded:
        if frame % 30 == 0:
            print(f"[Frame {frame}] dist={distance:.3f}, surface_gap={surface_gap:.3f}")

        if surface_gap < weld_surface_epsilon:
            print(
                f"[Frame {frame}] WELD CONDITION MET! "
                f"dist={distance:.3f}, surface_gap={surface_gap:.3f}"
            )
            joint_prim = weld_rigid_bodies(big_cube_path, small_cube_path)
            welded = True
            weld_frame = frame
            print(f"[Frame {frame}] >>> CUBES ARE NOW WELDED RIGIDLY ATTACHED <<<")

    if welded and not impulse_applied and frame >= weld_frame + 30:
        print(
            f"[Frame {frame}] Applying upward velocity "
            f"({upward_speed} m/s) to small_cube."
        )
        small_cube.set_linear_velocity(make_vel(0.0, 0.0, upward_speed))
        impulse_applied = True

    world.step(render=True)

simulation_app.close()

[RandomOakForest]

Thank you for following up. I'll move this issue to our Discussions for follow-up. I haven't actually tested the following, but maybe you could give this a try. Below is a minimal example of attaching two rigid bodies at runtime using the PhysX AutoAttachment API, which is supported in current Isaac Sim 4.x/5.x and works cleanly with Isaac Lab because it operates on USD/PhysX schemas instead of deprecated scene interfaces.¹²³

Example: attach two rigid bodies at runtime

Assumptions:

• You already have two rigid bodies in the stage at paths /World/BoxA and /World/BoxB.
• You call create_attachment_between_boxes() at some point during simulation (e.g., from a Kit script or an Isaac Lab task when both boxes are in contact / desired relative pose).²¹

# Works in Isaac Sim 4.x/5.x and is compatible with Isaac Lab stages.
# Uses only documented APIs from pxr and PhysX USD schemas.

from pxr import Usd
from omni.usd import get_context
from omni.physx.scripts import physicsUtils  # optional, but commonly used and documented [web:11]
from pxr import PhysxSchema

def create_attachment_between_boxes(
    actor0_path: str = "/World/BoxA",
    actor1_path: str = "/World/BoxB",
):
    # Get current USD stage
    stage: Usd.Stage = get_context().get_stage()

    actor0_prim = stage.GetPrimAtPath(actor0_path)
    actor1_prim = stage.GetPrimAtPath(actor1_path)
    if not actor0_prim or not actor1_prim:
        raise RuntimeError(f"Rigid body prim not found at {actor0_path} or {actor1_path}")

    # (Optional but recommended) ensure both prims have PhysX rigid body / collider APIs.
    # physicsUtils helpers are documented and used throughout Isaac Sim examples. [web:11][web:45]
    physicsUtils.add_rigid_body(stage, actor0_prim)
    physicsUtils.add_rigid_body(stage, actor1_prim)

    # Define a PhysxPhysicsAttachment under actor0
    attachment_path = actor0_prim.GetPath().AppendElementString("attachment_to_boxB")
    attachment = PhysxSchema.PhysxPhysicsAttachment.Define(stage, attachment_path)

    # Set the two actors to be attached
    attachment.GetActor0Rel().SetTargets([actor0_prim.GetPath()])
    attachment.GetActor1Rel().SetTargets([actor1_prim.GetPath()])

    # Let PhysX automatically compute the joint frames from current poses.
    # This is the key "easy" step: no manual joint transform math. [web:12][web:41]
    PhysxSchema.PhysxAutoAttachmentAPI.Apply(attachment.GetPrim())

    print(f"[INFO] Created attachment at {attachment_path} between {actor0_path} and {actor1_path}")

Usage pattern during simulation:

• After a physics step, when /World/BoxA and /World/BoxB are where you want them to “lock” together, call:

create_attachment_between_boxes("/World/BoxA", "/World/BoxB")

• PhysX will create a constraint that keeps the two rigid bodies moving together like a rigid assembly while preserving their current relative pose, without snapping, because the auto-attachment computes frames from the current transforms.³¹
• To “release” them, remove the attachment prim:

stage = get_context().get_stage()
stage.RemovePrim("/World/BoxA/attachment_to_boxB")

This is standard USD prim removal and is supported in all current Isaac Sim / Isaac Lab setups.²

Notes for Isaac Lab integration

• In Isaac Lab, you typically have a World/InteractiveScene managing the stage; the same code works as long as you call get_context().get_stage() or use World.instance().scene.stage as in the official examples.⁴³
• For robot–object grasping, just replace /World/BoxA and /World/BoxB with the rigid link prim path (e.g., a gripper link collision prim) and the object prim path; the attachment logic remains identical.¹³

Footnotes

1. https://simulately.wiki/docs/snippets/isaac-sim/deformable/ ↩ ↩² ↩³ ↩⁴

2. https://docs.isaacsim.omniverse.nvidia.com/4.2.0/simulation_fundamentals.html ↩ ↩² ↩³

3. https://forums.developer.nvidia.com/t/dynamically-create-attachments-at-runtime/322378 ↩ ↩² ↩³ ↩⁴

4. https://isaac-sim.github.io/IsaacLab/main/_modules/isaaclab/sim/schemas/schemas.html ↩

[xdd7561]

Thanks for the reply, this fixed my issue. From a bit of testing for my usecase, it seems like the links no longer snap back on GPU following your reply. I will reopen if the issue pops up again in other situations.