v0.7.1

Full Changelog: v0.7.0...v0.7.1

v0.7.0

This release adds a new Endoluminal Workflow, expands Agentic surgical simulation coverage, reorganizes the NVSkills agent platform, and introduces Rheo deformable-cloth simulation.

Highlights

• Added the Endoluminal Workflow for GPU-accelerated fluoroscopy simulation, CT-derived digital twins, catheter navigation, Slang DRR/DSA rendering, and ./i4h CLI integration.
• Expanded Agentic Workflow support from five to eleven registered environments, including six new dVRK/STAR surgical environments.
• Moved the agent skill catalog to top-level skills/, added NV-BASE validation artifacts, and introduced AGENTS.md, TESTING.md, and a local-agent/ runner.
• Added Rheo tablecloth-spreading simulation with surface-deformable cloth assets, Newton/PhysX backend support, XR teleop recording, and a dedicated cloth Docker image.

Endoluminal Workflow

• Added workflows/catheter_navigation/ for endovascular catheter navigation development.
• Added Slang-based differentiable DRR rendering with Beer-Lambert catheter compositing, batched rendering, DSA support, and detector realism.
• Added CT preprocessing, vessel segmentation, meshing, and centerline extraction support through pinned package dependencies.
• Added interactive guided catheter viewport support and i4h modes for preprocess_ct, segment_vessels, render_drr, and interactive_viewport.
• Added catheter navigation tests, Dockerfile, workflow metadata, README documentation, and seven new catheter-navigation skills.

Agentic Workflow

• Added six surgical Arena environments: surgical_reach_psm, surgical_reach_dual_psm, surgical_reach_star, surgical_lift_block, surgical_lift_needle, and surgical_lift_needle_organs.
• Added dVRK PSM, dual PSM, STAR, and ECM robot configs.
• Added scripted state-machine rollout support for surgical, scissor, and ultrasound environments.
• Added a GR00T N1.5 surgical baseline inference path for policy/Arena/Zenoh smoke validation.
• Improved policy routing, readiness checks, bridge-port discovery, headless E2E rendering, and vLLM annotator modes.

NVSkills Agent Platform

• Relocated canonical skills from .claude/skills/i4h-workflow-* to top-level skills/.
• Added benchmark files, skill cards, eval datasets, and skill.oms.sig artifacts for skill validation.
• Added AGENTS.md as the agent entry point and TESTING.md for offline validation guidance.
• Added local-agent/ tooling for running skills with local SGLang or NVIDIA-hosted inference.
• Removed i4h-workflow-dataset-transfer from the shipped skill catalog.

Rheo

• Added a tablecloth-spreading task with G1 29DoF + Inspire hands and Fourier H2 + Sharpa Wave hands teleop environments.
• Added switchable newton and physx physics backends.
• Added XR teleop recording via record_demos_tablecloth.py.
• Added Dockerfile.rheo_cloth to isolate cloth dependencies.
• Improved G1 Inspire finger tracking and hand-cloth collision behavior.

v0.6.0

What's Changed

[0.6.0] - Agentic Workflow, Claude Code Skills, SO-ARM Starter on GR00T N1.7

• Agentic Workflow: New unified IsaacLab-Arena + GR00T/openpi pipeline with five pre-trained environments, covering teleop recording, mimic expansion, VLM annotation, LeRobot conversion, fine-tuning, and rollout validation from a single CLI.
• Claude Code Agent Skills: Composable .claude/skills/i4h-workflow-* skills that drive the agentic workflow from natural-language prompts, from env creation and scene editing through end-to-end smoke runs.
• SO-ARM Starter on GR00T N1.7: SO-ARM workflow upgraded to GR00T N1.7 with refreshed DGX Spark / Jetson Thor / Jetson Orin container images.

Agentic Workflow

New workflows/agentic/ workflow unifying IsaacLab-Arena, GR00T (N1.5/N1.6/N1.7), and openpi PI0 behind YAML-driven dispatch.

See Agentic Workflow README.

Claude Code Agent Skills

Composable skills under .claude/skills/i4h-workflow-* covering setup, env creation, scene editing, dataset capture/curation, finetuning, validation, end-to-end runs, and LeRobot visualization. Includes documented scene-edit bridge recipes (live teleport, USD edits, camera bake-to-YAML) and hybrid env recipes (e.g. G1 + scissor scene) with pre-resolved component choices.

SO-ARM Starter

• GR00T N1.7 Upgrade: SO-ARM Starter workflow updated to GR00T N1.7.
• Container Refresh: DGX Spark, Jetson Thor, and Jetson Orin Dockerfiles upgraded.

Full Changelog: v0.5.0...v0.6.0

v0.5.0

[0.5.0] - New Rheo workflow, I4H CLI, and StreamLift

• Rheo Workflow: New end-to-end workflow for smart hospital automation and Physical AI development, featuring digital twin composition, expert demonstration capture, synthetic data generation, GR00T policy training with RL post-training, and pre-deployment validation.
• I4H CLI: Unified command-line interface across Robotic Surgery, Robotic Ultrasound, and SO-ARM Starter workflows, streamlining Docker builds, asset downloads, and workflow execution.
• StreamLift for Telesurgery: GPU-accelerated 4K image upsampling and downsampling operators for low-latency, high-resolution video streaming in telesurgery pipelines.
• Repository Restructure: Tutorials moved to a separate repository; improved layout, consolidated linting, and updated asset paths.

Rheo Workflow

New comprehensive workflow for autonomous clinical environment development, built on NVIDIA Isaac Lab and Isaac Lab Arena.

• Digital Twin Composition: Rapid environment assembly using Isaac Lab-Arena for OR-scale task composition and Isaac Lab for task-centric, manager-based environments with curriculum design and large-scale RL.
• Expert Demonstration Capture: Teleoperation via Meta Quest Controls for loco-manipulation tasks (surgical tray pick-and-place, case cart pushing) and precision bimanual manipulation (trocar assembly). Keyboard teleoperation is also supported for loco-manipulation tasks.
• Synthetic Data Generation: Simulation-driven data amplification with Isaac Lab Mimic/SkillGen-style pipelines, combined with Cosmos Transfer 2.5 guided generation for cross-scene generalization.
• Policy Training: Supervised fine-tuning of GR00T N1.5/N1.6 VLA models on curated datasets, with online RL post-training (PPO via RLinf) for precision manipulation tasks such as multi-step trocar assembly.
• Pre-Deployment Validation: Closed-loop policy evaluation runners with WebRTC camera streaming and trigger-based action execution for system-level verification.
• VLM Agents: Configurable VLM-powered agents for peri-operative annotation, surgical monitoring, robot control, and user command handling, with automated setup scripts.
• TensorRT Support: GR00T N1.6 TensorRT acceleration for Arena-based tasks.

Isaac for Healthcare Command Line Interface (I4H CLI)

Unified ./i4h command-line interface to simplify setup and execution across workflows. Workflows using I4H CLI now favor containerized development rather than setting up Conda environments on the host system.

• Robotic Surgery: CLI support for Docker build, asset download, and simulation launch.
• Robotic Ultrasound: CLI support for state machine, teleoperation, and evaluation modes with camera runtime configuration.
• SO-ARM Starter: Full CLI integration for simulation, teleoperation recording, policy training, and real-world deployment on DGX Spark, Jetson Orin, and Jetson Thor; simplified HDF5 recording path arguments; non-root simulation execution.
• Performance: Faster asset downloads by excluding blob data from CLI download steps.

StreamLift for Telesurgery

• 4K UpSampling/DownSampling: New GPU-accelerated Holoscan operators (C++ with Python bindings) for real-time 4K image upsampling and downsampling in telesurgery video pipelines.
• DGX Spark Support: Added workflow container support for DGX Spark platform.
• IGX Orin (CUDA 12): Real-world telesurgery workflow supported on IGX Orin.

Other Workflow Updates

• Robotic Ultrasound: Unified container environment for GR00T N1 and Pi0; re-enabled raysim; removed Cosmos Transfer 1 (placeholder for Cosmos Transfer 2.5); improved documentation and Quick Start guide with i4h CLI commands.
• SO-ARM Starter: Optimized x86_64 Dockerfile; added DGX Spark Isaac Sim container support and optimized DGX Dockerfile; aligned Jetson Thor environment to DGX; fixed Jetson Orin Dockerfile; improved documentation and Quick Start guide.
• Robotic Surgery: Updated README to streamline demo experience with i4h CLI commands.
• Repository: Improved layout and directory structure; added markdown linting; merged linting configs to root; updated IsaacSim 5.1 and IsaacLab compatibility fixes.

v0.4.0

What's Changed

• Upgrade IsaacSim 5.0 and IsaacLab 2.3 for surgery workflow
• Add holoscan support for so_arm_starter workflow
• Enhance documentation for so_arm_starter workflow
• Upgrade isaacsim5.0 isaaclab2.3 for Telesurgery workflow
• Add documentation and example of Thor and Origin deployment so_arm_starter workflow
• Upgrade to IsaacSim 5.0 IsaacLab2.3 for robotic ultrasound workflow
• Add DGX support for so_arm_starter workflow
• Update SE3 teleop isaaclab api change
• Fix pybind11 installation for Ubuntu 22 compatibility
• Improve Cosmos Transfer documentations
• Ignore .dat/.lic files to avoid adding RTI license to repo

Full Changelog: isaac-for-healthcare/i4h-workflows-internal@v0.3.0...v0.4.0

v0.3.0

What's Changed

• Add SO-ARM starter workflow with GR00T N1.5 integration and Sim2Real documentation
• Telesurgery: Update patient side to start in perspective view
• Supports HSB + imx274 camera as input source
• i4h CLI based on Holohub CLI
• Support for AJA Operator in Telesurgery
• Add YUAN HSB support
• Support auto-downloading models from Hugging Face

v0.2.0

What's Changed

• Fix dependency installation by @mingxin-zheng in #133
• Add Playback Recording Feature by @KumoLiu in #125
• Docker build for robotic surgery to enable webrtc streaming by @mingxin-zheng in #128
• Eval GR00T N1 model by @KumoLiu in #106
• Include cosmos transfer1 dependency by @KumoLiu in #134
• Make individual asset accessible by @mingxin-zheng in #126
• Improve documentation by @jjomier in #143
• Fix Cosmos Installation by @mingxin-zheng in #144
• Add docker build for robotic ultrasound by @mingxin-zheng in #131
• integrate cosmos-transfer1 + guided generation to I4H workflows by @guopengf in #129
• Add Virtual Incision MIRA teleoperate tutorial by @yiheng-wang-nv in #142
• Fix robot us docker build by @mingxin-zheng in #147
• Enhances sim_with_dds.py with HDF5 reset support and add a trajectory metrics evaluation pipeline. by @KumoLiu in #137
• Refactor lerobot format conversion and upgrade Isaaclab to v2.1.0 by @KumoLiu in #148
• allow choice of ultrasound probes matching raysim by @maximilianofir in #146
• Update MIRA tutorial to match up with new assets by @yiheng-wang-nv in #154
• Tutorial: Teleoperation with OpenXR Hand Tracking by @tbirdso in #145
• Enhances the test log printing with improved real-time output streaming by @mingxin-zheng in #160
• Telesurgery Workflow by @SachidanandAlle in #149
• Update default pi0 ckpt_path and documentation for the new weights by @KumoLiu in #162
• Temporary Solution for Segmentation Fault During Evaluation by @KumoLiu in #165
• Port main to release branch as preparation for v0.2.0rc1 by @mingxin-zheng in #163
• Refactor Holoscan Operators by @wendell-hom in #169
• Update transform_matrix and rot and pos factor in teleop with OpenXR by @KumoLiu in #161
• Update Instructions for Telesurgery Workflow by @mocsharp in #167
• Improve documentation by @jjomier in #173
• Update "Observations and Conclusion" part in evaluation folder by @KumoLiu in #166
• Pin the Isaac-GR00T repository to the n1-release branch and update related documentation and setup scripts by @KumoLiu in #178
• Update documentation by @jjomier in #179
• Fix Failure to Start Simulation LiveStream from Remote Docker Container by @mingxin-zheng in #175
• Rewrite the robotic surgery Dockerfile to fix livestream issues by @mingxin-zheng in #182
• Post release candidate 0.2.0rc2 by @mingxin-zheng in #183
• Update Pi0 and GR00TN1 model weights and Readme by @KumoLiu in #184
• Comprehensive documentation refractor for robotic ultrasound workflow by @mingxin-zheng in #181
• Refactoring of the robotic surgery documentation to break out subfolders into focused READMEs by @mingxin-zheng in #185
• Improve Telesurgery documentation by @mingxin-zheng in #186
• Update telesurgery from VDR feedback by @wendell-hom in #188
• Update tags for v0.2.0rc3 for several packages by @mingxin-zheng in #187
• Address ssh key feedback from VDR by @wendell-hom in #189
• Post v0.2.0rc3 to release by @mingxin-zheng in #190
• Add what's new for v0.2.0 by @KumoLiu in #193
• Update Dependency Tags to 0.2.0 and Remove SSH Token Use by @mingxin-zheng in #192
• Post v0.2.0 to release by @mingxin-zheng in #194

New Contributors

• @guopengf made their first contribution in #129
• @tbirdso made their first contribution in #145

Full Changelog: v0.1.0...v0.2.0

v0.2.0rc3

What's Changed

• Update Pi0 and GR00TN1 model weights and Readme by @KumoLiu in #184
• Comprehensive documentation refractor for robotic ultrasound workflow by @mingxin-zheng in #181
• Refactoring of the robotic surgery documentation to break out subfolders into focused READMEs by @mingxin-zheng in #185
• Improve Telesurgery documentation by @mingxin-zheng in #186
• Update telesurgery from VDR feedback by @wendell-hom in #188
• Update tags for v0.2.0rc3 for several packages by @mingxin-zheng in #187
• Address ssh key feedback from VDR by @wendell-hom in #189
• Post v0.2.0rc3 to release by @mingxin-zheng in #190

Full Changelog: 0.2.0rc2...v0.2.0rc3

0.2.0rc2

What's Changed

• Refactor Holoscan Operators by @wendell-hom in #169
• Update transform_matrix and rot and pos factor in teleop with OpenXR by @KumoLiu in #161
• Update Instructions for Telesurgery Workflow by @mocsharp in #167
• Improve documentation by @jjomier in #173
• Update "Observations and Conclusion" part in evaluation folder by @KumoLiu in #166
• Pin the Isaac-GR00T repository to the n1-release branch and update related documentation and setup scripts by @KumoLiu in #178
• Update documentation by @jjomier in #179
• Fix Failure to Start Simulation LiveStream from Remote Docker Container by @mingxin-zheng in #175
• Rewrite the robotic surgery Dockerfile to fix livestream issues by @mingxin-zheng in #182
• Post release candidate 0.2.0rc2 by @mingxin-zheng in #183

Full Changelog: v0.2.0rc1...0.2.0rc2

v0.2.0rc1

What's Changed

• Fix dependency installation by @mingxin-zheng in #133
• Add Playback Recording Feature by @KumoLiu in #125
• Docker build for robotic surgery to enable webrtc streaming by @mingxin-zheng in #128
• Eval GR00T N1 model by @KumoLiu in #106
• Include cosmos transfer1 dependency by @KumoLiu in #134
• Make individual asset accessible by @mingxin-zheng in #126
• Improve documentation by @jjomier in #143
• Fix Cosmos Installation by @mingxin-zheng in #144
• Add docker build for robotic ultrasound by @mingxin-zheng in #131
• integrate cosmos-transfer1 + guided generation to I4H workflows by @guopengf in #129
• Add Virtual Incision MIRA teleoperate tutorial by @yiheng-wang-nv in #142
• Fix robot us docker build by @mingxin-zheng in #147
• Enhances sim_with_dds.py with HDF5 reset support and add a trajectory metrics evaluation pipeline. by @KumoLiu in #137
• Refactor lerobot format conversion and upgrade Isaaclab to v2.1.0 by @KumoLiu in #148
• allow choice of ultrasound probes matching raysim by @maximilianofir in #146
• Update MIRA tutorial to match up with new assets by @yiheng-wang-nv in #154
• Tutorial: Teleoperation with OpenXR Hand Tracking by @tbirdso in #145
• Enhances the test log printing with improved real-time output streaming by @mingxin-zheng in #160
• Telesurgery Workflow by @SachidanandAlle in #149
• Update default pi0 ckpt_path and documentation for the new weights by @KumoLiu in #162
• Temporary Solution for Segmentation Fault During Evaluation by @KumoLiu in #165
• Port main to release branch as preparation for v0.2.0rc1 by @mingxin-zheng in #163

New Contributors

• @guopengf made their first contribution in #129
• @tbirdso made their first contribution in #145

Full Changelog: v0.1.0...v0.2.0rc1