v1.4.145

What's Changed

• ci: keep prerelease wheels off PyPI via Kitmaker mirroring_strategy (1.4.x) by @aristarkhovNV in #1070
• [release/1.4.x] fix(webxr): constrain React to the Fiber-compatible 19.2 minor by @yanziz-nvidia in #1092
• [1.4] Fix docs dependencies after switching to main configuration by @yanziz-nvidia in #1096

Full Changelog: v1.4.142...v1.4.145

v1.4.142

Release Notes — Isaac Teleop 1.4

Everything that landed in release/1.4.x since release/1.3.

Streaming & CloudXR

The CloudXR runtime launches in-process, so examples no longer need one started for them

#700, #703, #748, #749, #750, #751, #695, #776

CloudXRLauncher grew argparse helpers and a launch_context(args) entry point, and every shipped example now uses them: python example.py brings up the runtime and WSS proxy itself instead of requiring a pre-started runtime or a sourced environment. --no-launch-cloudxr-runtime attaches to an existing one, and --cloudxr-install-dir / --cloudxr-device-profile / --cloudxr-env-config / --accept-eula are available uniformly. The device profile defaults to Quest3. If you embed Isaac Teleop, this is the pattern to copy — one with CloudXRLauncher.launch_context(args) around your session.

CloudXR moves to 6.3.0, with link quality visible to the operator

#1025, #973, #895, #768, #857

Both the Runtime SDK and the Web SDK are on 6.3.0 (from 6.2.0). The bump is API-compatible — everything new is additive — and the client adopts the diagnostics it brings: a 4-bar session-quality indicator on the in-XR HUD, and a Pose Send FPS card that measures the uplink command rate. That second number is what separates "the video is choppy" from "the robot is not receiving my commands", which previously could not be told apart from inside the headset.

The experimental runtime ships alongside the stable one as cloudxr_exp

#808, #975, #889, #940

Wheels now carry a second runtime, isaacteleop.cloudxr_exp, next to the stable one, selected with ISAAC_TELEOP_CLOUDXR_EXP. On Tegra T234 the experimental runtime is preferred automatically because it is the one that performs there; ISAAC_TELEOP_CLOUDXR_EXP=0 forces stable. Source builds still default the bundle off, since the experimental tarball lives on private NGC until a version goes GA.

Ctrl+C tears down the runtime and OpenXR session in the right order

#816, #966, #770

Shutting down a CloudXR-backed session used to produce a flood of Broken pipe and XR_ERROR_RUNTIME_FAILURE errors and could leave a container unusable: the runtime process died before the OpenXR handles it served were released. The signal handler now propagates to the prior handler first so with-block teardown unwinds inside-out, and OpenXRSession releases its handles deterministically at context-manager exit rather than at garbage collection. SIGTERM, SIGINT and atexit all stop a session-scoped runtime, and main-thread join is selected automatically on Orin.

The WSS proxy can be turned off

#757

If you already run your own secure WebSocket proxy, the launcher's built-in one can now be disabled instead of failing with a port-in-use error when an example starts. Useful for any deployment that terminates TLS ahead of Isaac Teleop.

Devices, Plugins & Trackers

reBot DevArm leader arm plugin, with Damiao and RobStride backends

#729

The Seeed reBot DevArm (6-DOF plus gripper) streams as JointStateOutput on the same generic joint-space path the SO-101 leader plugin uses, so existing joint-space consumers pick it up unchanged. Both hardware builds are supported and inferred from the device argument: a path selects the Damiao dm-serial bus, a bare SocketCAN interface name such as can0 selects RobStride. Neither backend needs a vendor SDK, and a synthetic fallback lets the plugin run headless in CI.

OGLO tactile glove device

#691

A complete new device: BLE client, packet parser, plugin, tracker, schema and example. It follows the standard plugin layout, so it is also a reasonable reference if you are adding a BLE glove of your own.

Vendor-neutral haptic feedback, with Manus gloves as the first consumer

#568

Device output now has a transport, not just device input. A generic HapticCommand schema moves over XR_NVX1_push_tensor via TensorPushTracker / HapticCommandReaderTracker, so any out-of-process haptic device — glove, exoskeleton — reuses one wire format instead of adding a per-vendor tracker and FlatBuffer. Manus glove haptics is the first device wired through it.

Generic SE3 tracker device type

#778

Any 6-DoF pose source can now be a first-class device without a bespoke schema. Se3TrackerPose is built on core.Pose, with units, Hamilton (x, y, z, w) quaternion order, reference-frame rules and two-level validity documented normatively in the schema itself. The tracker stack mirrors the joint-state pattern (live and replay implementations, recording traits, Python bindings), and an example controller plugin shows the producer side.

Full-body tracking is vendor-neutral, with the vendor chosen at session level

#781

The full-body schema and tracker dropped their Pico-specific names, and the vendor is now selected per source on the session rather than baked into the type you instantiate. Recordings are unaffected — the wire format and enum values are identical. See Breaking Changes for the rename and for the new replay-mode restriction.

The Manus plugin exposes raw sensor data

#796, #719

Alongside the retargeted skeleton, the plugin can surface the glove's raw sensor values, which is what you want for calibration work, glove-side debugging, or training on unprocessed signals. Naming was standardized on "MANUS" across code and docs at the same time.

Rendering & Visualization

ProjectionLayer brings full-view RGBD content to Televiz

#544

Televiz can now display full-view RGBD content — gsplat, nvblox, neural reconstruction — through a new ProjectionLayer, with color and depth composited so quad layers Z-composite correctly against it. The frame loop was reworked in the same change: VizSession::begin_frame now returns the real per-eye XR poses from xrLocateViews instead of placeholder identity views, so a renderer that renders against info.views is pose-correct by construction. Renderers that relied on the old placeholder behavior were rendering against the wrong pose and should be re-checked.

QuadLayer can submit native OpenXR quad layers

#855

Opt in with QuadLayer::Config::use_openxr_quad_layer and the quad is submitted as a real XrCompositionLayerQuad instead of being composited into the shared render target. When every visible layer is a native quad, the shared projection layer is dropped entirely, which lets the runtime engage its quad fast path and client-side reconstruction — the difference shows up as sharper text and lower latency on streamed UI panels. Window and offscreen modes ignore the flag.

Televiz builds by default and installs from PyPI

#720

BUILD_VIZ now auto-detects: on when Vulkan, the CUDA Toolkit and glslangValidator are present, off otherwise, so a core-only source build still configures. More importantly, camera_viz setup installs isaacteleop from PyPI — the published wheel bundles the viz module — so trying Televiz no longer requires a source build at all. Source builds become an opt-in development path.

camera_viz gains video-file replay, an XR-first default, and opt-in RTP

#766, #918, #945, #721

A new type: video frame source plays a recording through the normal FrameSource contract, so the whole camera-streaming path can be exercised with no camera attached — including the NVENC/RTP chain — and stereo: true splits side-by-side recordings per eye on the GPU. Display now defaults to XR rather than a desktop window, and RTP became opt-in via --with-rtp so direct-mode setup no longer pays for the GStreamer and PyGObject build (both are behavior changes; see Breaking Changes). A Dell 4MP webcam V4L2 config ships for desktop testing.

Client & Web UI

Record and replay XR input from the browser client

#765, #814, #812

The web client can capture a scoped XR input trace and play it back, with a trace visualization and a recorder UI. Replay is time-paced by default, with the older frame-paced mode still available, so a...

v1.3.131

This release expands Isaac Teleop from a CloudXR/WebXR teleoperation core into a multi-device, multi-robot platform: new dexterous-hand and arm retargeting, data gloves and exoskeletons, a ROS 2 reference integration, ARM64 support, a record/replay data pipeline, a preview Vulkan visualization stack, and a substantially streamlined CloudXR operations story.

---

Top 10 new features

1. SO-101 arm teleoperation — leader arm + XR controller (LeRobot ecosystem)

Two complementary ways to drive the low-cost SO-101 5-DOF arm:

• Generic joint-space device + SO-101 leader arm (#660): a reusable device path (name-keyed JointStateOutput schema → JointStateSource → JointStateRetargeter) with joint (leader→follower mirror) and ee_pose (URDF forward-kinematics) modes — reusable for any joint-encoder device (leader arms, exoskeletons, gloves). The reference so101_leader plugin reads the 6 Feetech STS3215 servos over serial with no SDK dependency, wire-compatible with LeRobot's FeetechMotorsBus, and ships a calibrate subcommand that mirrors lerobot-calibrate (and can emit LeRobot-format calibration files).
• SO-101 XR-controller retargeters (#651, hardened by #692): drive an SO-101 follower from an XR controller in Isaac Lab (Isaac-Stack-Cube-SO101-IK-Abs-v0) — clutch-rebased absolute EE pose (engaging never jerks the arm) + proportional analog-trigger gripper; #692 gates the clutch on grip-pose validity to prevent NaN/IK crashes on tracking loss.

Maturity: the leader / joint-space device is a complete reference implementation (plugin, retargeter, calibration tool, docs, example, MCAP-replay backend); the SO-101 XR-controller retargeters are experimental (tune-in-sim). No LeRobot package dependency — SO-101 is a LeRobot-ecosystem arm; a separate examples/lerobot dataset integration predates this release.

2. Sharpa Wave dexterous-hand retargeters

Map OpenXR hand tracking onto the Sharpa Wave hand via two interchangeable backends (#449, #474, #484, #490, #491): dexpilot (default for hand teleop) wrapping the open-source dex-retargeting DexPilot optimizer with SHA-pinned official URDFs, and pink_ik — a Pinocchio/Pink IK path ported from V2D (robotic_grounding). Bimanual combiners and ROS hand-mode integration with output joint-name aliasing.
Maturity: mixed — dexpilot runs on open-source deps; pink_ik requires V2D's robotic_grounding (bundled as a temporary bridge). dexpilot emits a fixed 22-joint set; pink_ik's DOF count is model-dependent.

3. Teleoperation with Isaac ROS (ROS 2 reference) — teleop_ros2

A single-node rclpy publisher (ROS 2 Humble) that streams controller, hand, head, end-effector, locomotion, and retargeted finger-joint data over standard topics. Four selectable modes (controller_teleop, hand_teleop, controller_raw, full_body); standard ROS remapping (#343); foot-pedal locomotion (#395); configurable TF frames, finger-joint names, and config/asset roots (#368, #507); in-process CloudXR via CloudXRLauncher (#609); head-pose publishing (#618); deterministic MCAP-fixture integration tests.
Maturity: preview. This ships under the Isaac ROS umbrella (the web-client picker has an isaacros entry; the quick start has a "Teleoperation with Isaac ROS" card pointing here) as a ROS 2 reference publisher; the broader Isaac ROS CUDA-accelerated framework remains tracked as "Planned."

4. Egocentric Hand Reconstruction (codename "Ego4Robo")

A post-processing pipeline (src/postprocessing/egocentric_hand_reconstruction, #363/#417/#672/#599) that recovers world-frame 4D hand + camera motion from a single egocentric video by chaining two containerized models — ViPE (camera pose/depth) → Dyn-HaMR (MANO hand mesh). Exports per-track MANO meshes/joint trajectories, "jumpy-track" quality control, and an OSMO cloud-batch workflow — turning head-cam video into demonstration data for imitation learning.
Maturity: experimental (v0.0.1; requires user-supplied licensed MANO/BMC assets). User-facing name is "Egocentric Hand Reconstruction"; "Ego4Robo" is the internal codename / CI branch prefix.

5. ARM64 / aarch64 support

Isaac Teleop now installs and runs on ARM64 hosts such as NVIDIA DGX Spark (#506, #516, #509, #513, #358, #589, #681): the ROS 2 reference container auto-builds the nlopt wheel (none on PyPI for aarch64), required-environments platform constraints, Manus-plugin arch-conditional SDK fixes, and a Python 3.13 + arm64 CI leg.
Maturity: first-class for the containerized ROS 2 / CI path; bare-metal pip install of the [retargeters] extra still needs a documented one-time nlopt wheel build. (Jetson Nano remains planned.)

6. Streamlined CloudXR operations

• CloudXRLauncher (#353): programmatic, in-process start / health_check / stop of the CloudXR runtime + WSS TLS proxy (context-manager API; runs the runtime as an isolated subprocess to avoid CUDA-context clashes with Isaac Sim). Replaces the now-deprecated run_cloudxr_via_docker.sh (#440) and powers ROS 2 live teleop (#609).
• Out-of-band (OOB) teleop setup (#388, #390, #415, #438, #461, #655): --setup-oob auto-opens the web client on a USB-tethered headset over ADB and clicks CONNECT for you via Chrome DevTools Protocol; optional --usb-local relays signalling/media over the USB cable with a local coturn TURN server; a side-channel metrics/control hub; configurable ports.
• CloudXR runtime and web SDK bumped to 6.2.0 (#564, #528).

Maturity: mixed (Launcher GA; OOB automation newer). Note: OOB = out-of-band, not out-of-box — the control hub runs on a side channel beside the video stream. --usb-local still requires Wi-Fi associated (Chromium's WebRTC excludes loopback ICE candidates), though no teleop traffic crosses Wi-Fi.

7. Televiz (preview)

A new opt-in (BUILD_VIZ default OFF) Vulkan + OpenXR + CUDA compositor exposed to Python as isaacteleop.viz (#423, #427, #451, #466, #471, #502, #517, #535, #536, #542, #641). VizSession composites GPU camera frames into XR / window / offscreen targets via QuadLayer (mono or true stereo), with zero-copy CUDA↔Vulkan interop (DeviceImage). Ships a camera_viz example driving v4l2 / OAK-D / ZED / synthetic sources (plus a RealSense 435i config, #683), locally or over RTP H.264, and can share one OpenXR session with TeleopSession.
Maturity: preview — QuadLayer only today; ProjectionLayer / 3D rendered content explicitly not yet shipped. Product name: "Televiz" (capital T, lowercase v).

8. MCAP record & replay

Record any teleop session — head, hands, controllers, full-body (Pico), pedals, message channels — to an MCAP file (the open Foxglove standard, with FlatBuffer payloads), then replay it headless through the exact same retargeting pipeline (SessionMode.REPLAY), frame-aligned and deterministic, with no headset attached (#290, #330, #389, #396, #401–#404, #425, #508, #527, #554). Ideal for CI/regression, debugging, and dataset generation; includes viser browser replay examples for hand/controller/full-body.
Maturity: mixed — recording is GA-quality; replay has preview edges (no end-of-file signal; recordings made before #527 must be re-recorded for correct frame alignment).

9. Manus data-glove plugin (partner-authored)

Integrates MANUS Core SDK gloves for full hand-joint tracking (#431, #441, #510, #526, #513, #520, #525). The wrist anchor auto-selects between HMD optical hand tracking (XR_MNDX_xdev_space) and a Quest 3 controller fallback; per-finger poses are injected into the OpenXR hand layer so any retargeter consumes them transparently. Includes installer/udev scripts (SHA-256-verified SDK), a Vulkan "MANUS Data Visualizer", and a ROS controller-mode path feeding the Sharpa retargeters.
Maturity: mixed (Linux/x86_64 primary, ARM fixed; proprietary SDK not redistributed). Authored by Manus — see Community & partner contributions.

10. Web client overhaul

The browser teleop client (now simply the "web client", renamed from "WebXR client" in #671) becomes the default entry point (#338, #398, #420, #473/#476, #537, #556/#560, #561, #591, #629, #671, #685, #687):

• Registry-driven application picker including Isaac Lab, Isaac Sim, ROS, and GEAR.
• Every CloudXR setting is URL-settable via a param registry; settings persist per application.
• Headless mode + window.xrDevice exposure for programmatic / AI-agent control.
• Per-version hosting (/client/<slug>/, /client/stable/) and self-hosting via --host-client.
• Robustness: old-browser warning, self-signed-cert acceptance link, stale-WebGL-error draining, exit-on-streaming-error, XR-panel drag fix, Server-IP clear button.
  Maturity: mixed.

---

Other notable improvements

• OpenXR controller haptic feedback (rumble) (#566) — a vendor-neutral device-output path (HapticSink / IHapticDevice → xrApplyHapticFeedback) so sim contact force or a trigger pull can vibrate the operator's controller. GA path with example + tests; the broader force-feedback framework (EndEffectorForce) is foundation ahead of consumers.
• Haptikos hands plugin (#486, #642) (partner — Magos) — an OpenXR plugin for the Haptikos hand exoskeleton (hand/finger pose injection only; Linux / Meta Quest). See Community section.
• Pipelined retargeting (#467) — opt-in RetargetingExecutionMode.PIPELINED runs the retarget graph on a background worker so step() returns the latest finished result instead of blocking the control loop, with latency telemetry and deadline pacing.
• Teleop state machine (#340, #391) — DefaultTeleopStateManager (stopped/paused/running + reset), fail-safe to STOPPED on input loss, with reset wired into stateful retargeters.
• Camera pipeline (#323, #266, #349, #341, #203) — correct full-rang...

v1.1.8

What's Changed

• Backport webclient versioned publish 1.1.x by @yanziz-nvidia in #465
• backport: Clear WebGL errors on startup (#537) to release/1.1.x by @yanziz-nvidia in #563
• ci: backport per-tag web client publish + stable seed to 1.1.x by @yanziz-nvidia in #558

Full Changelog: v1.1.5...v1.1.8

v1.1.5

What's Changed

• fix: ensure 'isaacteleop' is installed from the local dir (#284) by @jiwenc-nv in #301
• fix: Bypass body tracker if not supported by the runtime by @life1ess in #297
• fix(retargeting): DLPack copy order and optional transform tests by @nvddr in #386
• cherrypick: automated workflow for release tag by @jiwenc-nv in #455

Full Changelog: v1.0.191...v1.1.5

Release 1.0.193

Hotfix

Fix

• Packaging: Relaxed requires-python in the isaacteleop Python wheel from ==<build-python>.* to >=3.10 (#405).

Why

Wheels built against different Python minor versions (e.g. 3.10 vs 3.11) were each pinning requires-python to their exact build-time version. This made the wheels non-interchangeable on PyPI/NGC and prevented users on a different Python minor from installing a published wheel that would otherwise work.

Impact

• pip install isaacteleop now resolves correctly across supported Python 3.10+ interpreters regardless of which interpreter the wheel was built against.
• No API or runtime behavior changes; packaging metadata only.
• Single-file change: src/core/python/pyproject.toml.in.

Release 1.0.191

Isaac Teleop 1.0.191 Release Notes

Release Date: March 16, 2026

This is a the first GA release that marks the transition from Isaac Teleop 0.3 to 1.0, representing the product's first stable public release. It includes 204 commits spanning 488 changed files with ~42,700 additions and ~13,200 deletions since v0.3.15.

---

Highlights

• Rebranded to Isaac Teleop -- the project has been officially renamed to Isaac Teleop.
• Python wheel-based installation -- Isaac Teleop can now be installed via pip install as a Python wheel, replacing the Docker-first workflow as the default setup.
• ROS 2 integration -- a full-featured ROS 2 Reference App (RA) with support for ROS Humble and Jazzy, teleop modes, TF tree publishing, and CycloneDDS.
• Full body tracking -- added PICO XR full body tracker, finger joint retargeting, and full-body mode support across the ROS pipeline.
• CloudXR 6.1.0 -- upgraded to CloudXR Runtime 6.1.0 and WebXR client 6.1.0.
• ARM64 support -- builds and CI now support ARM64 architecture for both the core runtime and Docker images.
• Comprehensive documentation -- new Sphinx-based docs covering quick start, device tracking, teleop sessions, retargeters, build-from-source, and licensing.

---

New Features

ROS 2 Reference App

• Added ROS 2 Reference App for teleoperation (teleop_ros2) with support for both Humble and Jazzy distributions.
• Introduced ROS teleop modes for structured control switching.
• Added ability to publish left and right wrist transforms to the TF tree.
• Publish controller data in full_body mode to support seamless mode transitions.
• Encapsulated publisher logic in a dedicated ROS 2 node.
• Added optional CycloneDDS dependency for ROS teleop.

Body & Hand Tracking

• Added PICO XR full body tracker (FullBodyTrackerPico).
• Added finger joint retargeting.
• Full body tracking support in the ROS 2 Reference App.
• Added hand tracking directions and controller aim pose fallback when hand tracking is unavailable.
• Made head pose optional in the retargeter source.
• Created tracked types with optional data to handle inactive input sources.

Retargeters

• Implemented foot pedal retargeter with hot-plug support for Linux joystick API devices.
• Added tunable parameters to the SE3 retargeter.
• Added transform utilities, TransformMatrix type, and .transformed() methods.
• Added optional values to the retargeter system.
• Added retargeter timing instrumentation.
• Moved retargeters under src/ for cleaner project structure.

Camera & Streaming

• New Holoscan-based camera streamer sample application.
• Added SSH push command for cam_streamer with exposed camera outputs.
• Camera streamer reconnection fix using re-exec pattern.
• Multi-stream and sensor check for OAK camera.
• Camera tracker now supports multiple channels.
• Added distance threshold for camera plane repositioning.

Recording & Schema

• Implemented SchemaPusher & Tracker on top of Generic Tensor Device.
• Added lossless schema recording for tensor devices.
• Created SchemaRecord wrapper types for all schemas.
• Multi-channel recording support for hand and controller trackers.
• Added MCAP support for per-stream metadata.

CloudXR & Runtime

• Updated to CloudXR Runtime 6.1.0 (via rc1 through rc7 progression).
• Bundled CloudXR runtime into TeleopCore for simplified deployment.
• Added CloudXR Runtime SDK and Dockerfile.
• Added process management for CXR runtime and WSS proxy.
• Added runtime and WSS proxy script.
• Proxy shows a friendlier page when the SSL certificate is accepted.
• Fixed CloudXR runtime crash from OpenSSL symbol conflict.

Installation & Deployment

• Python wheel is now the default installation method (pip install isaacteleop).
• Added EULA acceptance via --accept-eula flag.
• Kitmaker integration for NVIDIA distribution.
• Added wheel repair step for release builds.

OpenXR

• Restructured to native_openxr architecture.
• Added support for passing external OpenXR handles to Teleop Session.
• Added xdev examples and inline utilities for loading extension functions.
• Added plugin_args support to plugin_manager.

Teleop Session

• Enabled external inputs in TeleopSession.
• Added collection_id parameter to the plugin.
• Updated timestamp handling with local and device clock.
• Improved GPU index configurability.

---

Bug Fixes

• Fixed quaternion ordering in dex retargeting and general retargeting.
• Fixed joystick input ordering in controller source.
• Fixed multiple controller trackers using temporary workaround.
• Fixed camera streamer reconnection with re-exec.
• Fixed camera streamer Docker build on ARM64.
• Fixed CloudXR runtime crash from OpenSSL symbol conflict.
• Fixed MCAP schema writer for OAK.
• Fixed package version handling.
• Fixed stubgen caching.
• Fixed script that assumed docker compose exists.
• Handled numpy ABI for both 1.x and 2.x compatibility.
• Used numpy>=1.22.2 to address CVE-2021-41495.
• Fixed ROS 2 Teleop Dockerfile nlopt build on ARM64.
• Relaxed nlopt version constraint for x86.

---

Platform & Infrastructure

• Enabled ARM64 builds and CI testing.
• Added Windows workflow concurrency settings.
• Upgraded to Ubuntu 22.04 for CI builds.
• Bumped GitHub Actions (checkout, cache, setup-python, artifact actions) to latest versions.
• Pinned uv version for reproducible builds.
• Pre-installed heavy Python dependencies in Docker images to speed up launches.
• Enabled CodeRabbit automated code review.
• Set up ruff pre-commit and REUSE-compliant license checking.

---

Documentation

• Complete Sphinx-based documentation site with multi-version support.
• Quick start guide for wheel-based installation.
• Guides for teleop sessions, device tracking, retargeters, and the device ecosystem.
• Build-from-source reference (replacing the old BUILD.md).
• SSL certificate setup visuals.
• ROS documentation updated to match implementation.
• Added copy button, dark mode fixes, and improved code block styling.
• Source code linking to GitHub from docs.

---

Breaking Changes

• Renamed from previous project name to Isaac Teleop -- all package references updated.
• Retargeters moved under src/ -- import paths have changed.
• Restructured to native_openxr -- OpenXR code reorganized.
• run_cloudxr.sh now uses the wheel-based flow by default (Docker-based flow available as run_cloudxr_via_docker.sh).
• IDeviceIOSource refactored -- added poll_tracker() and updated class hierarchy.

---

Known Dependencies

• Python 3.x with numpy>=1.22.2
• CloudXR Runtime SDK 6.1.0
• ROS 2 Humble or Jazzy (for teleop_ros2)
• Ubuntu 22.04 & 24.04 (Linux)
• No Windows support (basic build validation only)

Release 0.3.15

Isaac Teleop (TeleopCore) Release Notes

Release 0.3.15 (vs 0.2)

Release branch: release/0.3.x

---

Highlights

• Project rename: TeleopCore is now Isaac Teleop — the unified framework for high-fidelity ego-centric and robotics data collection. Documentation and branding have been updated accordingly.
• Retargeting framework: Port of Isaac Lab retargeters with a type system, source nodes, and examples (gripper, SE3, dex bimanual, G1 tri-hand, locomotion, reordering). Retargeter tuning UI with IMGUI for live tuning.
• Camera & recording: Generic Camera Device Plugin interface; Oak-D plugin; MCAP recorder integrated into deviceio session; generic per-frame metadata in camera schema.
• Full body & locomotion: Pico full body tracking schema and testbed; locomotion schema and bindings compatible with Isaac OS Lower Body Controller.
• CloudXR: Runtime updates to 6.1.0 (pid3/pid4/rc2); tensor extension support; streamlined Docker setup and env handling; Quest3 as default device profile.
• Python & packaging: Python 3.10, 3.11, and 3.12 support; PEP 440 versioning; multi-version wheel builds in CI; git commit count in versioning.

---

New Features

• Retargeting

  • Retargeter type system and interface; port of Isaac Lab retargeters (SE3, gripper, dex, combined).
  • Source nodes and enums for source node identifiers.
  • Examples: controller + SE3, gripper, dex bimanual, G1 tri-hand, locomotion, full bimanual reordering.
  • Retargeter tuning UI (IMGUI) for live tuning.
  • Optional retargeting dependencies in build.

• Camera & device plugins

  • Generic Camera Device Plugin interface; Oak-D camera plugin (DepthAI) for recording.
  • MCAP recorder integrated into deviceio_session with configurable recording path; Python example for session with MCAP recording.
  • Generic per-frame metadata in camera.fbs (#88).
  • hands.fbs for tracking both hands to align with hand tracker logic.

• Full body & locomotion

  • Data schema and bindings for Pico full body tracking.
  • Body tracking testbed for Pico WebXR body tracking extension (cxrjs/bodytracking example).
  • Locomotion schema and bindings compatible with Isaac OS Lower Body Controller operator.

• Session & OpenXR

  • Create OpenXRSessionHandles from Python (#81).
  • Teleop session manager package: helpers, config, and teleop_session_example.
  • Shared core::OpenXRSession reuse across plugins; OpenXR math helpers moved to oxr_utils.
  • CXR tensor extension headers (Generic Tensor Collection / push tensor) for plugins.

• Device I/O & schemas

  • Device output for Generic3AxisPedalOutput (e.g. Logitech-style pedals).
  • Timestamps on device data; controller tracker uses FlatBuffer schema.
  • Tensor and TensorGroup classes with type validation and tests; tensor types for retargeting engine I/O.

• Build & developer experience

  • Devcontainer configuration for Isaac Teleop (#66).
  • Teleop utils for building teleop apps.
  • TELEOPCORE_DISABLE_CXR_ENV_CHECKS to disable CloudXR env checks when needed.

---

Improvements

• DeviceIO & trackers

  • Tracker creation refactor: initialize → create_tracker; implementations retrieved from session; removed is_initialized() from tracker interfaces (#33, #34).
  • Device plugin reuses controller from deviceio; rate-limited warnings in DeviceIOSession to reduce log spam.
  • deviceio C++ files renamed to snake_case for consistency.

• CloudXR

  • CloudXR env setup: fail early on missing env; use NV_CXR_RUNTIME_DIR (replacing XDG_RUNTIME_DIR where appropriate); ensure OpenXR env vars (XR_RUNTIME_JSON, etc.) are set in oxr_session.
  • CloudXR Docker: run_cloudxr.sh can download Web SDK from NGC, build containers, and start services; cloudxr_image_ops.sh for flexible image load/save/pull with container_tag.
  • Migrated CloudXR JS SDK download from manual PID to NGC CLI; one-time NGC setup documented.
  • README improvements for CXR Runtime setup; HAProxy Dockerfile tidied (permissions).

• Build & CI

  • Hunter cache (.hunter) to improve build speed for OAKD dependencies (#85).
  • ccache (Ubuntu) and sccache (Windows) support (#29).
  • CMake cleanup: redundant checks removed, inc aligned with target names, deviceio_py_utils library (#40).
  • DepthAI/CMake: policy version set for compatibility with CMake ≥ 4.0 (#60).
  • Ubuntu CI uses Ubuntu-22.04; GPU runner for GitHub Actions; native tests run in CI.
  • Publish-wheel job for tagged releases (#70); multi-Python wheel builds (#89).
  • Docker-based test fixture for CloudXR tests (local and CI); run_tests_with_cloudxr.sh improvements and documentation.

• Versioning & deps

  • Single consolidated VERSION file; version includes git commit count (#83); bump to 0.3.x.
  • PEP 440 versioning for alpha, rc, dev, local, and final releases (#87).
  • NumPy 1.22+ (aligned across docs); optional update to 2.4.0 in examples.
  • get_home_dir improved (Windows: USERPROFILE; runtime error instead of /tmp fallback).

• Security & supply chain

  • npm install runs with --ignore-scripts to reduce supply chain risk (#9da1d58).
  • HAProxy config ownership set to root to address security hotspot.

• Oak-D / camera

  • Default size switched to match camera resolution to avoid unnecessary scaling and preserve FOV (#74).

• LeRobot

  • Recorder fixes; dataset written relative to recorder.py location.

• Misc

  • OpenXR SDK compatibility via sanitizers-cmake (#43).
  • Removed internal references (#72); README and doc cleanups.
  • Issue templates updated; unnecessary README sections removed.
  • UV: specify Python version in Dockerfile.test; remove uv.lock files from repo.

---

Bug Fixes

• Use get_required_extensions correctly for OpenXR.
• Fix typo in load_cloudxr_images.sh.
• Remove duplicate file and ctor definitions from OpenXRSessionHandles; add default initializers (#67).
• LeRobot recorder fixes and path handling.

---

API / Behavior Changes

• Trackers: initialize() replaced by create_tracker(); is_initialized() removed from tracker interfaces and implementations.
• Versioning: VERSION file consolidated; format follows PEP 440; git commit count used in version string.
• CloudXR: Prefer NV_CXR_RUNTIME_DIR; stricter env checks unless disabled via TELEOPCORE_DISABLE_CXR_ENV_CHECKS.
• Plugin utils: Controllers and session helpers removed from plugin_utils (logic moved/reused in deviceio/plugins where applicable).

---

Dependencies & Requirements

• Python: 3.10, 3.11, and 3.12 supported.
• NumPy: 1.22 or newer (documented; 2.4.0 used in some examples).
• CloudXR: Runtime 6.1.0 series (pid3, pid4, rc2, or webrtc patch as referenced in commits).
• Build: Ubuntu 22.04 for build-ubuntu workflow; CMake policy compatibility for DepthAI with CMake ≥ 4.0.

---

Known Limitations / Notes

• Oak-D plugin: Windows build skipped in CI due to ffmpeg-related constraints (see #60).
• CloudXR tests require GPU and optional Docker/CloudXR setup; see run_tests_with_cloudxr.sh and README.

---

Generated from diff: release/0.2..release/0.3.x (TeleopCore repo).

Release 0.2.0

Overview

This release introduces new features including a device plugin system, data schemas
with FlatBuffer serialization, and major improvements to the hand/head tracking systems. The
codebase has been modernized with C++20 support, improved Python packaging, and better code
quality through clang-format enforcement.

---

🚀 New Features

Plugin Manager System

• Plugin Manager API - New plugin architecture for discovering and managing external plugin
  binaries with YAML-based metadata configuration
• Plugin health monitoring - Added check_health API that throws PluginCrashException if
  the plugin process crashes
• Manus Glove Plugin - Full integration with Manus SDK for glove-based hand tracking with
  controller pose fusion
• Controller Synthetic Hands Plugin - Generate synthetic hand poses from VR controllers

Schema & Data Serialization

• FlatBuffer-based schemas for efficient data serialization:
  • pose.fbs - Pose representation with position and quaternion
  • tensor.fbs - Tensor message for multi-dimensional data
  • hand.fbs - Hand pose schema with joint data
  • head.fbs - Head tracking schema
• Python bindings for all schemas with comprehensive unit tests
• C++ Object API for working with schema data

Device I/O Module

• Renamed and reorganized xrio module to deviceio for clarity
• Controller Tracker - New tracking support for VR controllers
• Head Tracker - Dedicated head pose tracking component
• Integrated FlatBuffer schemas for head and hand pose data

Examples & Documentation

• LeRobot Example - New example for robot learning data recording and visualization
• Camera Streaming Example - Standalone GStreamer-based camera streaming with OAK-D support
  (host/robot Docker configs)
• Improved README with better setup and build instructions

---

🔧 Improvements

Build System & Dependencies

• Switched from Git submodules to CMake FetchContent for dependency management
• Upgraded to C++20 to enable designated initializers
• clang-format enforcement in CI for consistent code style
• Improved Python packaging:
  • Locked Python version in pyproject.toml
  • Explicit Python version and ABI in wheel filenames
  • Auto-generated .pyi stub files for Python bindings
• Use mainline yaml-cpp instead of fork

Code Quality

• RAII improvements - Replaced OpenXRHandle with std::unique_ptr
• RAII for XR core functions - Better resource management
• Namespace reorganization - Renamed from oxr to core
• Extensive refactoring of hand tracker and session APIs

CI/CD

• Added clang-format checks to Linux builds
• CI optimizations to avoid exceeding usage limits
• Fixed Python test paths in Ubuntu build workflow

CloudXR Integration

• Enabled push device support in CloudXR container
• Updated web app and WSS proxy Dockerfiles
• Run nginx with non-root user for security
• Fixed HAProxy configuration permissions

---

⚠️ Breaking Changes

• xrio module renamed to deviceio - update import paths accordingly
• TeleopSession API replaced with DeviceIOSession
• Namespace changed from oxr:: to core:: in C++ code

---

🙏 Contributors

Thanks to all contributors who made this release possible!

Release 0.1.0

Initial commit for Isaac Teleop Core 0.1.0