Repository navigation
Release Notes — Isaac Capture v1.6
Highlights of changes since v1.5. Full comparison.
Isaac Capture v1.6 adds Xsens and SOMA body/hand input, keyboard and steering-wheel control, and automatic recovery for interrupted WebXR sessions. Device-health monitoring and an end-effector pose guard help applications detect input loss and hold tracking jumps. The Python package moves to isaaccapture; see Breaking Changes before upgrading.
Devices & Input
Xsens MVN suits provide full-body input
#1083 (partner contribution — Movella/Xsens)
Stream Xsens MVN suit poses into existing body-tracking applications without an XR headset. The Linux x86_64/aarch64 plugin receives MVN Studio's UDP stream and publishes the standard 24-joint layout through body.xsens.
SOMA body and hand input supports retargeting and visualization
Use body and hand poses from the SOMA-X model in retargeting and visualization. Sources accept joint rotations or joint positions and orientations, and the DeviceIO viewer lets users select body and hand sources independently. See the SOMA integration guide for model setup and supported inputs.
Sparse joint poses support partial tracking without invented joints
JointSe3PoseOutput represents only the joints a device actually measures, keyed by joint name. Missing joints mean untracked. The MANUS integration publishes its five fingertip poses through this interface, with live visualization and MCAP recording/replay examples.
Keyboard input integrates with recording, replay, and retargeting
Control planar motion, relative 3D motion, and grippers from a keyboard, with MCAP recording and replay. The in-process tracker preserves short key presses and releases held keys when its input surface loses focus. GLFW and terminal examples are included.
Steering-wheel and pedal input drive a vehicle-control example
The vehicle-control reference example uses a Logitech G923 steering wheel and pedals on Linux, with keyboard fallback and command logging/replay. It sends commands over ZMQ to a Panda worker and publishes neutral before exiting on wheel disconnect or stale samples. Other wheels are not guaranteed to work; vehicle-side integration requires hardware-specific setup. See the example setup and limitations.
- MANUS SDK dependency updated from 3.2.0 to 3.2.1. (#1131)
CloudXR Runtime & Connection Recovery
CloudXR Runtime SDK moves to 6.4.0
The CloudXR Runtime SDK is updated from 6.3.0 to 6.4.0. The Web SDK remains at 6.3.0.
Recover WebXR sessions after stream errors and USB interruptions
#1122, #1127, #1142, #1145, #1146
Recover interrupted sessions with configurable retries and stream-start timeouts. The client detects sessions that enter XR without receiving video. After USB interruptions, the out-of-band (OOB) launcher repairs transport and tries to reuse the browser session before restarting browser setup.
Startup reports readiness and checks the USB connection
See which startup stage is blocking a connection. USB-local startup checks link health, confirms recovery from fresh streaming evidence, and resumes when the headset wakes.
Launcher-started services host the matching web client by default
The launcher hosts the matching CloudXR.js assets through the WSS proxy and reports configuration mismatches when attaching to an existing service. Standalone service CLI hosting remains explicit.
Session Health, Retargeting & Diagnostics
TeleopSession exposes provider and device health
Distinguish a running provider from a connected physical device through TeleopSession status snapshots. Monitoring covers OpenXR and supported plugins, including MANUS. Headset disconnects and reconnects are detected even without a head tracker in the application pipeline.
An end-effector pose guard holds tracking discontinuities
Applications can add EePoseJumpGuard to their retargeting graph to hold the last accepted pose during linear tracking jumps or missing/invalid input. Enforcement applies only while execution is RUNNING; recovery from a rejected jump requires consecutive nearby samples. Adding the guard is an application integration step, not an automatic effect of upgrading. Rate limiters also expose diagnostic status.
Python and C++ share one logging system
Troubleshoot sessions with consistent logs across Python, C++, and launched processes. A shared API controls console output and log files, and applications can capture output from native libraries during a selected operation.
ROS 2 can express end-effector poses relative to the head
The ROS 2 reference example adds a head-relative end-effector pose frame for integrations that need operator-relative commands.
🪲 Bug Fixes
Sessions stay connected while waiting for an application
Decoder warm-up timeouts are disabled by default so idle runtime frames do not trigger a disconnect while waiting for an OpenXR application to start. Setting a timeout to 0 now correctly disables detection instead of firing immediately. Stream-attach detection remains enabled by default.
Kill and input-loss events reset teleoperation correctly
Holding kill no longer emits a reset every frame. Input loss now triggers a reset even when the reset control disappears with the run/kill controls, while inputs absent from startup do not count as a loss.
Sharpa Avatar gloves recover after glove or dongle disconnection
#1197, #1198, #1381 (partner contribution — Sharpa in #1381)
Left and right gloves recover independently after power cycling or USB dongle reconnection. Health reporting distinguishes transport, glove, and dataset failures, while advancing sample timestamps prevent cached data from appearing live. Stale haptic commands are stopped and are not replayed after reconnection.
Retargeting installs no longer miss the six dependency
The retargeting dependencies now explicitly include six, avoiding missing-dependency failures when using dex-retargeting.
MANUS stops injecting stale skeletons and handles metadata failures
The plugin no longer continues injecting stale skeleton data after updates stop and handles skeleton-metadata failures.
Multi-GPU visualization selects and reports the correct CUDA device
Projection-layer submission makes the visualization CUDA device current on the calling thread, fixing worker-thread failures on multi-GPU machines. Applications can inspect the session's CUDA device, and camera streaming reports CUDA mismatches and source failures more clearly.
The WebXR panel remains reachable and teardown exits sessions cleanly
Head-relative reset and tracking fix missing or unreachable headset panels; the Performance panel renders correctly. Visualization teardown requests OpenXR session exit, and the OOB client disconnects when its owning program quits.
⚠️ Breaking Changes
Python packages and examples move to isaaccapture
Install and depend on isaaccapture; update imports and python -m commands accordingly. The isaacteleop runtime alias remains available until 1.9, but does not provide full static-type compatibility. The example namespace changes immediately to isaaccapture_examples with no old-name alias. Update logger configuration strings to isaaccapture.* as well.
Upgrade an installation with CloudXR support:
pip install -U "isaaccapture[cloudxr]"Update imports and example commands:
-import isaacteleop
+import isaaccapture
-python -m isaacteleop_examples.teleop_session_manager.teleop_session_example
+python -m isaaccapture_examples.teleop_session_manager.teleop_session_exampleThe tmux rig launcher is removed
isaacteleop.rig and its bundled rig YAML configurations are removed. Start the CloudXR service and the required producer/consumer processes through their documented standalone workflows.
SO-101 and reBot asset-directory overrides are removed
Custom launch scripts must stop passing the removed asset-directory overrides and use the built-in asset handling.