Skip to content

Release v0.4.0

Latest
Latest

Choose a tag to compare

View all tags
@shalin-a-jain shalin-a-jain released this 23 Sep 22:02
· 32 commits to main since this release
v0.4.0
33129dd
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Highlights

V2D v0.4.0 introduces FlashCHORD for accelerated GPU-parallel robotic reference-tracking training for loco-manipulation and dexterous manipulation, alongside improvements to reconstruction tools, robotic grounding, and closed-loop GR00T evaluation.

FlashCHORD (NEW!)

Accelerated policy training from human-object motion using Newton, Warp, and JAX.

Summary

  • Training recipes: adds FlashSAC and PPO training in Newton for floating Sharpa, Vega Sharpa, and G1 + Dex3.

  • Complete pipeline: documented setup, robot and scene inspection, kinematic and dynamic reference replay, training, evaluation, and policy viewer.

  • Motions and scenes: consumes the same parquet format as already released robotic grounding, existing motion references can directly be used with FlashCHORD.

  • Configuration: documents rewards, terminations, tracking tolerances, resets, curriculum, randomization, and debug-marker controls.

  • Execution: provides a dedicated Docker environment, local training progress, and OSMO templates for training and evaluation.

Known assumptions

  • The documented full training recipes use a 48 GiB GPU with at least 46,000 MiB free, Linux, NVIDIA driver 580 or newer, Docker, NVIDIA Container Toolkit, and Git LFS.

Known limitations

  • FlashCHORD uses its own container and training entrypoints. The existing robotic grounding and end-to-end GR00T workflow continues to use Isaac Lab. Support for Isaac Lab 3.0 with Newton backend will come in later releases.

Docs: robotic_grounding/flash_chord/README.md

Robotic Grounding and End-to-End Workflow

Summary

  • Motion quality: adds signed-distance-field penetration checks for concave and hollow objects, with convex-hull fallback when no cached field is available.

  • Parallel evaluation: uses tiled wrist cameras for Vega Sharpa.

Reconstruction

Summary

  • Texture editing: adds an optional logo-redaction workflow with model-proposed regions, manual review, and synchronized GLB/PNG texture updates that preserve mesh geometry and UVs.

  • Mesh quality: adds input-file hashes, aligned dimensions, and surface-based scale estimates to geometry comparison reports.

Known limitations

  • Ego reconstruction (DynHaMeR, HaMeR, and HaMeR_mesh) may exhibit inconsistent object scale, orientation flips, and hand–object misalignment, particularly for nearly symmetric objects or automatically generated meshes. Results may vary between runs due to non-deterministic camera trajectory estimation and mesh generation.

  • CARI4D preserves the scale of the supplied object mesh. Meshes generated with SAM3D image-to-mesh must be scaled before being passed as --object_mesh_path; CARI4D does not automatically apply the separately estimated SAM3D transform.

Docs: object reconstruction · CARI4D · egocentric reconstruction

Assets 2
Loading