Skip to content

Bring in Sharpa retargeter from V2D#449

Merged
jiwenc-nv merged 3 commits intomainfrom
sharpa_with_v2d
May 6, 2026
Merged

Bring in Sharpa retargeter from V2D#449
jiwenc-nv merged 3 commits intomainfrom
sharpa_with_v2d

Conversation

@jiwenc-nv
Copy link
Copy Markdown
Collaborator

@jiwenc-nv jiwenc-nv commented May 1, 2026

Summary by CodeRabbit

  • New Features

    • Added Sharpa hand retargeter for converting OpenXR hand tracking input to robot hand joint angles
    • Added optional [grounding] extra package with hand retargeting capabilities and dependencies
    • Added demo script showcasing Sharpa hand retargeting in live and synthetic modes
  • Documentation

    • Added Sharpa hand retargeter setup and usage guide with configuration details
  • Tests

    • Added integration tests validating Sharpa hand retargeter functionality

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 1, 2026

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: cda7eb13-406e-423f-bcf4-fcc0fdde16c7

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

This pull request introduces an optional [grounding] extra that bundles the V2D robotic_grounding package into Isaac Teleop wheels for Sharpa hand retargeting. Changes include a GitHub Action that conditionally fetches and caches V2D source keyed by commit SHA, CMake build configuration to stage the package and propagate a BUNDLE_ROBOTIC_GROUNDING flag, a Python wheel sanitization script to strip bundled content from public Release artifacts, implementation of SharpaHandRetargeter and SharpaBiManualRetargeter classes that perform IK-based hand joint conversion, supporting test and demo scripts, documentation, and dependency requirements files specifying Pinocchio-based solver tooling.

Sequence Diagram(s)

sequenceDiagram
    participant GHA as GitHub Actions
    participant Cache as GitHub Cache
    participant CLI as GitHub CLI (gh)
    participant Repo as jiwenc-nv/v2d
    participant FS as Filesystem
    participant CMake as CMake Build

    GHA->>GHA: Read deps/v2d/version.txt
    GHA->>GHA: Extract pinned SHA
    GHA->>Cache: Lookup cache key v2d-src-{sha}
    
    alt Cache Hit
        Cache-->>GHA: Return cached deps/v2d/src
    else Cache Miss
        GHA->>CLI: gh repo clone jiwenc-nv/v2d (retargeter)
        CLI->>Repo: Clone retargeter branch
        Repo-->>CLI: Repository content
        GHA->>GHA: git checkout {pinned-sha}
        GHA->>FS: Copy robotic_grounding/ to deps/v2d/src/
        GHA->>FS: Clean __pycache__ directories
        GHA->>Cache: Store in cache with key v2d-src-{sha}
    end
    
    GHA->>GHA: Check for __init__.py
    GHA->>GHA: Set bundled output true/false
    GHA->>CMake: Pass -DBUNDLE_ROBOTIC_GROUNDING={bundled}
    CMake->>FS: Copy robotic_grounding into wheel staging
    CMake->>FS: Generate wheel with package
Loading
sequenceDiagram
    participant Build as Build System
    participant CMake as CMake
    participant Wheel as Wheel File
    participant Script as strip_robotic_grounding.py
    participant Artifact as Release Artifact

    Build->>CMake: Configure with BUNDLE_ROBOTIC_GROUNDING
    CMake->>Wheel: Create wheel (robotic_grounding included)
    
    alt Release Branch
        Build->>Script: Call strip_robotic_grounding_from_wheel.py
        Script->>Wheel: Scan for robotic_grounding/* entries
        Script->>Wheel: Create temp wheel without matched entries
        Script->>Wheel: Recompute SHA-256 hashes & RECORD
        Script->>Wheel: Atomically replace original
        Script->>Build: Report modifications made
    else Non-Release Branch
        Wheel->>Artifact: Use as-is (robotic_grounding included)
    end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 45.16% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Bring in Sharpa retargeter from V2D' directly and accurately reflects the main change—integrating the Sharpa hand retargeter from the V2D repository into the codebase.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch sharpa_with_v2d

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

  • Generate code and open pull requests
  • Plan features and break down work
  • Investigate incidents and troubleshoot customer tickets together
  • Automate recurring tasks and respond to alerts with triggers
  • Summarize progress and report instantly

Built for teams:

  • Shared memory across your entire org—no repeating context
  • Per-thread sandboxes to safely plan and execute work
  • Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started


Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.github/actions/setup-v2d-src/action.yml:
- Around line 46-52: The current check only ensures deps/v2d/version.txt is
non-empty; update the script to validate that the value in SHA is a full git
commit SHA (e.g. match /^[0-9a-f]{40}$/i) before writing to GITHUB_OUTPUT and
proceeding, and fail with an explicit error if it doesn't match; apply the same
SHA regex validation to the counterpart script (scripts/setup_v2d_src.sh) so
both the action and the setup script reject branch/tag names and only accept
commit SHAs.

In `@scripts/strip_robotic_grounding_from_wheel.py`:
- Around line 63-76: The strip step currently only removes files under
robotic_grounding/ and rewrites RECORD; extend it to also rewrite the package
METADATA inside the .dist-info directory to remove the grounding extra: locate
the METADATA entry when iterating zin.infolist() (use the same loop that
references item.filename, kept_records and record_arcname), read and parse its
text, drop any "Provides-Extra: grounding" lines and remove "[grounding]"
suffixes from any Requires-Dist entries (e.g., turn "isaacteleop[grounding]"
into "isaacteleop"), then write the modified METADATA back into zout and add its
new RECORD line via _record_line so RECORD reflects the changed METADATA before
you append the final RECORD row and write RECORD using kept_records and
record_arcname.

In `@src/retargeters/__init__.py`:
- Around line 105-121: The current __init__.py exposes SharpaHandRetargeter,
SharpaBiManualRetargeter, and SharpaHandRetargeterConfig unconditionally which
breaks wheels built without the robotic_grounding bundle; update the module to
only add these three symbols to the export mapping when the same bundling signal
used at build time is present (e.g. a module-level flag like
_HAS_ROBOTIC_GROUNDING or a guarded import check for the robotic_grounding
package), otherwise omit/hide those entries and replace the pip-install error
text with a gated message that appears only when the bundle is available;
specifically modify the export dict population in __init__.py so
SharpaHandRetargeter, SharpaBiManualRetargeter, and SharpaHandRetargeterConfig
are only added when the bundling flag/import check succeeds.

In `@src/retargeters/sharpa_hand_retargeter.py`:
- Around line 284-291: The loop that builds
_output_indices_left/_output_indices_right and _left_indices/_right silently
prefers left when a joint name appears in both left_joint_names and
right_joint_names; update the logic in the loop over target_joint_names (the
block that appends to _output_indices_left/_output_indices_right and
_left_indices/_right) to detect the overlap (jname present in both
left_joint_names and right_joint_names) and reject it instead of choosing one
side — e.g., raise an exception or log an explicit error and skip/abort the
mapping for that jname so duplicate names cannot silently misroute data.
- Around line 237-244: Wrap the call to self._kinematics.compute(...) in a
try-except inside the retargeter method so any exception from compute does not
abort the frame; on exception call self._emit_zeros() and return (or set
new_qpos to zeros) and do not update self._qpos_prev, otherwise continue using
the returned result and update self._qpos_prev = new_qpos.copy(). Locate the
compute call in the method that invokes self._kinematics.compute and ensure the
exception handler mirrors the existing invalid-input pattern (calls
_emit_zeros() as lines handling invalid inputs do) and only updates
self._qpos_prev on successful compute.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 59566bda-8c3d-4d35-8849-9c02c7aae965

📥 Commits

Reviewing files that changed from the base of the PR and between a939dd1 and ceb85ce.

📒 Files selected for processing (19)
  • .github/actions/setup-v2d-src/action.yml
  • .github/workflows/build-ubuntu.yml
  • .gitignore
  • deps/v2d/version.txt
  • docs/source/index.rst
  • docs/source/references/retargeting/index.rst
  • docs/source/references/retargeting/sharpa.rst
  • examples/retargeting/python/sharpa_hand_retargeter_demo.py
  • scripts/setup_v2d_src.sh
  • scripts/strip_robotic_grounding_from_wheel.py
  • src/core/python/CMakeLists.txt
  • src/core/python/pyproject.toml.in
  • src/core/python/requirements-grounding.txt
  • src/core/python/requirements-retargeters.txt
  • src/core/retargeting_engine_tests/python/CMakeLists.txt
  • src/core/retargeting_engine_tests/python/pyproject.toml
  • src/core/retargeting_engine_tests/python/test_sharpa_hand_retargeter.py
  • src/retargeters/__init__.py
  • src/retargeters/sharpa_hand_retargeter.py

Comment thread .github/actions/setup-v2d-src/action.yml
Comment thread scripts/strip_robotic_grounding_from_wheel.py Outdated
Comment thread src/retargeters/__init__.py
Comment thread src/retargeters/sharpa_hand_retargeter.py
Comment thread src/retargeters/sharpa_hand_retargeter.py
@jiwenc-nv
Copy link
Copy Markdown
Collaborator Author

/preview-docs

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 1, 2026

✅ Preview deployed: https://NVIDIA.github.io/IsaacTeleop/preview/pr-449/

@jiwenc-nv jiwenc-nv requested a review from sgrizan-nv May 6, 2026 06:24
jiwenc-nv and others added 3 commits May 6, 2026 06:37
Note: V2D is still pre-release and gated from public access. The fetch
is gated on V2D_RETARGETER_TOKEN and skipped on release branches.
Wraps robotic_grounding.retarget.SharpaHandKinematics with Teleop's
BaseRetargeter contract -- OpenXR -> MANO joint mapping, warm-started
qpos, output indexing. Lazy-imports cleanly: installs without the
[grounding] extra raise a directed ModuleNotFoundError instead of
breaking module load.

Co-Authored-By: rwiltz <165190220+rwiltz@users.noreply.github.com>
Promotes references/retargeting.rst into a directory and adds
sharpa.rst covering [grounding] build, Python usage, the demo, and
ctest validation.
@jiwenc-nv jiwenc-nv enabled auto-merge (rebase) May 6, 2026 08:03
@jiwenc-nv jiwenc-nv merged commit c330462 into main May 6, 2026
62 of 70 checks passed
@jiwenc-nv jiwenc-nv deleted the sharpa_with_v2d branch May 6, 2026 09:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants