Add C++ full-body tracking examples by jiwenc-nv · Pull Request #807 · NVIDIA/IsaacTeleop

jiwenc-nv · 2026-07-19T23:37:41Z

Summary

Full-body tracking (FullBodyTrackerPico, XR_BD_body_tracking) was fully supported in the C++ DeviceIO API but only demonstrated from Python. This adds the missing C++ examples and wires them into the docs.

examples/schemaio/full_body_printer.cpp — minimal C++ reader for the 24-joint body skeleton, following the se3_printer pattern (DeviceIOSession + tracker, ~30 Hz print loop with per-joint is_valid gating).
examples/mcap_record_replay/cpp/record_full_body.cpp — C++ counterpart of record_full_body.py: records the full_body channel by passing a core::McapRecordingConfig to DeviceIOSession::run(). It uses the same channel base name as FullBodySource, so recordings replay unchanged with replay_full_body.py (record in C++, replay/visualize in Python).
Docs — device/body_tracking.rst, device/trackers.rst, and references/mcap_record_replay.rst now reference the C++ examples, including a note on C++-side MCAP recording.

Testing

Both targets build cleanly (full_body_printer, record_full_body).
clang-format-14 clean; SKIP=check-copyright-year pre-commit run --all-files passes.
Smoke-ran both binaries without a CloudXR runtime: they fail gracefully at OpenXR session creation with the standard NV_CXR_RUNTIME_DIR guidance, matching the sibling examples.
Replay compatibility of the channel naming verified against the topic layout of an existing Python-recorded MCAP (<source name>/<sub_channel>).

Summary by CodeRabbit

New Features
- Added a C++ example for reading and printing streamed full-body joint data.
- Added a C++ example for recording full-body tracking sessions to MCAP files.
- Added build and installation support for both examples.
Documentation
- Expanded full-body tracking and MCAP documentation with usage examples and replay guidance.
- Added references to corresponding C++ and Python workflows.

github-actions · 2026-07-19T23:37:52Z

📝 Docs preview is not auto-deployed for fork PRs.

A maintainer with write access to NVIDIA/IsaacTeleop can deploy a preview by
commenting /preview-docs on this PR. Once deployed, the preview
will live at:

https://nvidia.github.io/IsaacTeleop/preview/pr-807/

coderabbitai · 2026-07-19T23:38:01Z

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: c353735c-8eaa-417b-a211-62e05041991b

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

📝 Walkthrough

Walkthrough

Adds a C++ full-body pose printer and a C++ MCAP recorder. The examples create FullBodyTrackerPico and OpenXR sessions, update DeviceIOSession, and process full-body pose data. The recorder writes full_body topics, closes the MCAP writer on shutdown, and prints a replay command. CMake builds and installs both executables, while documentation links the examples and describes recording and replay.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant FullBodyTrackerPico
  participant OpenXRSession
  participant DeviceIOSession
  participant MCAPWriter
  participant replay_full_body.py
  FullBodyTrackerPico->>OpenXRSession: provide required extensions
  OpenXRSession->>DeviceIOSession: start session with full_body recording config
  DeviceIOSession->>MCAPWriter: write full_body topics
  DeviceIOSession->>DeviceIOSession: update until duration elapses
  MCAPWriter-->>replay_full_body.py: provide recorded full_body topics

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: adding C++ full-body tracking examples.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

coderabbitai

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@examples/schemaio/full_body_printer.cpp`:
- Around line 91-105: Bound the sampling loop in the full-body printer so it
cannot run indefinitely when tracked.data remains null. Add a wall-clock
deadline or maximum update count around the while loop, exit when the bound is
reached, and report that no body-tracking samples arrived while preserving the
existing sample-count termination and printing behavior.
- Around line 50-57: Update the sample output around the pelvis and head pose
retrieval to check each joint’s own is_valid() status before printing
coordinates. Print an unavailable marker for an invalid pelvis or head, while
preserving coordinate output independently for each valid joint; do not rely on
the aggregate valid_count or skeleton-level tracking flag.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 95415ff7-5082-44cc-b921-bd680e6ab51d

📥 Commits

Reviewing files that changed from the base of the PR and between 26821bb and 19ffc03.

📒 Files selected for processing (8)

docs/source/device/body_tracking.rst
docs/source/device/trackers.rst
docs/source/references/mcap_record_replay.rst
examples/mcap_record_replay/CMakeLists.txt
examples/mcap_record_replay/cpp/CMakeLists.txt
examples/mcap_record_replay/cpp/record_full_body.cpp
examples/schemaio/CMakeLists.txt
examples/schemaio/full_body_printer.cpp

aristarkhovNV

LGTM, I wonder if we could both print and record in one sample. Recording is like one extra line of setup.

- examples/schemaio/full_body_printer.cpp: minimal C++ reader for FullBodyTrackerPico (XR_BD_body_tracking, 24 joints), following the se3_printer pattern. - examples/mcap_record_replay/cpp/record_full_body.cpp: C++ counterpart of record_full_body.py; records the full_body channel by passing a McapRecordingConfig to DeviceIOSession::run(), producing files that replay unchanged with replay_full_body.py. - Reference both examples from the body-tracking, trackers, and MCAP record/replay docs. Signed-off-by: Jiwen Cai <jiwenc@nvidia.com>

Wire the full-body tracking C++ examples into the rig launcher so one command brings up the whole session instead of a hand-started runtime: - rigs/full_body.yaml: first consumer-only rig — no producer plugin and no collection_id; the printer (validity gate) and the MCAP recorder read XR_BD_body_tracking joints directly from the runtime. No params: the recorder keeps its own defaults, and a fresh timestamped .mcap is written per Enter-rerun. - full_body_printer: print '[body tracking inactive]' once per second while body tracking is unavailable (limp mode), sharing the literal record_full_body already uses, so the rig's validity-gate pane is never silently mute. - rig_tests: load + footgun tests for the shipped rig, mirroring the se3_tracker pair. - docs: rig launch pointers in body_tracking.rst and mcap_record_replay.rst, a limp-mode troubleshooting entry, and rig.rst now documents the consumer-only rig shape (collection_id rendezvous scoped to producer rigs). Build green (full_body_printer, record_full_body), rig tests 30/30, pre-commit and Sphinx docs clean. Signed-off-by: Jiwen Cai <jiwenc@nvidia.com>

Follow-up to the generic FullBodyTracker refactor (NVIDIA#781), which removed deviceio_trackers/full_body_tracker_pico.hpp and renamed the Pico types: FullBodyTrackerPico -> FullBodyTracker, FullBodyPosePicoT -> FullBodyPoseT, BodyJointPico_* -> BodyJoint_*. full_body_printer and record_full_body now use the new header and names; no behavior change (the session still selects the default vendor). Both targets build, rig tests 30/30, clang-format-14 and pre-commit clean. Signed-off-by: Jiwen Cai <jiwenc@nvidia.com>

- full_body_printer: gate the pelvis/head printout on each joint's own is_valid flag ([not tracked] marker instead of unspecified coordinates), and bound the read loop (2x the sample budget) so limp mode cannot spin forever — a zero-sample run now exits with a clear message. - Keep the C++ examples self-contained: drop all references to the Python examples from sources, comments, and the recorder's stdout (the rig file pointer stays; the two-ways-to-read-full-body story lives in the docs). - Drop CMAKE_DL_LIBS from both example targets: dl is propagated transitively by the OpenXR loader; verified by linking without it. Signed-off-by: Jiwen Cai <jiwenc@nvidia.com>

Running the full_body rig dropped full_body_<timestamp>.mcap into the repo root (the pane cwd), showing up as untracked noise in git status. - record_full_body: the output argument may now be a directory (existing or with a trailing slash) — it is created if needed and each run writes a fresh timestamped file inside it, so reruns still never clobber. - rigs/full_body.yaml: point the recorder at examples/mcap_record_replay/recordings/ — already gitignored and the default search location of the replay workflow, so a bare replay invocation picks up rig takes automatically. - body_tracking.rst: document where takes land. Verified all three output modes (dir arg creates the directory and timestamps inside; explicit .mcap path unchanged; bare default unchanged); rig tests 30/30; clang-format-14 and pre-commit clean. Signed-off-by: Jiwen Cai <jiwenc@nvidia.com>

aristarkhovNV

update the description to refer to generic full body tracker, pico naming is now deprecated

coderabbitai Bot reviewed Jul 19, 2026

View reviewed changes

Comment thread examples/schemaio/full_body_printer.cpp Outdated

Comment thread examples/schemaio/full_body_printer.cpp Outdated

jiwenc-nv force-pushed the jiwenc/deviceio-example branch from 19ffc03 to 9868fd6 Compare July 21, 2026 04:39

jiwenc-nv requested review from aristarkhovNV and nv-jakob July 21, 2026 04:41

aristarkhovNV approved these changes Jul 21, 2026

View reviewed changes

Comment thread examples/mcap_record_replay/cpp/CMakeLists.txt Outdated

Comment thread examples/mcap_record_replay/cpp/CMakeLists.txt Outdated

Comment thread examples/mcap_record_replay/cpp/record_full_body.cpp Outdated

jiwenc-nv added 3 commits July 22, 2026 17:30

jiwenc-nv force-pushed the jiwenc/deviceio-example branch from 9868fd6 to 654cf7d Compare July 22, 2026 17:41

jiwenc-nv added 2 commits July 22, 2026 17:56

aristarkhovNV approved these changes Jul 22, 2026

View reviewed changes

jiwenc-nv merged commit 0d5d178 into NVIDIA:main Jul 22, 2026
42 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add C++ full-body tracking examples#807

Add C++ full-body tracking examples#807
jiwenc-nv merged 5 commits into
NVIDIA:mainfrom
jiwenc-nv:jiwenc/deviceio-example

jiwenc-nv commented Jul 19, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

github-actions Bot commented Jul 19, 2026

Uh oh!

coderabbitai Bot commented Jul 19, 2026 •

edited

Loading

Review skipped

Walkthrough

Sequence Diagram(s)

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

Uh oh!

aristarkhovNV left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aristarkhovNV left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

jiwenc-nv commented Jul 19, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Testing

Summary by CodeRabbit

Uh oh!

github-actions Bot commented Jul 19, 2026

Uh oh!

coderabbitai Bot commented Jul 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Walkthrough

Sequence Diagram(s)

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

aristarkhovNV left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aristarkhovNV left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jiwenc-nv commented Jul 19, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jul 19, 2026 •

edited

Loading