[OvPhysX] Adds FrameView for the OVPhysX backend#5678
Conversation
There was a problem hiding this comment.
Code Review: Add OvPhysxFrameView for OVPhysX backend
Thanks for this contribution, @AntoineRichard! This PR adds important OVPhysX backend support to the FrameView abstraction. The implementation is well-structured and follows the established patterns from NewtonSiteFrameView.
✅ Strengths
- Clean architecture: The site-based approach with Warp kernels mirrors the Newton implementation appropriately
- Thorough documentation: Docstrings and limitation sections are comprehensive
- Deferred initialization: Properly handles the async physics setup via
PHYSICS_READYcallback - Test coverage: Good use of the shared contract test suite
- USD/Physics separation: The env_0-scoped export workaround is well-documented with clear limitations
📝 Suggestions
Documentation
-
Changelog accuracy (
changelog.d/ovphysx-frameview.minor.rst:6): The changelog mentions reading from "scene data provider'sbody_qarray", but the implementation uses a directRIGID_BODY_POSEtensor binding. Consider updating for consistency. -
Scale getter behavior (
ovphysx_frame_view.py:785): Theget_scales/set_scalesdocstrings could clarify behavior forclone_usd=Falsescenes whereenv_1..Nhave no USD prims.
Testing
-
World-attached sites: Consider adding a test for prims with no rigid body ancestor (the
WORLD_BODY_INDEX == -1branch). -
Rigid body rejection: The guard at lines 507-513 that rejects prims with
RigidBodyAPIdeserves explicit test coverage.
Minor Code Quality
-
Indexed query allocation (
ovphysx_frame_view.py:650): The per-callwp.zerosallocation in indexed queries is fine for correctness but worth noting in docs if these become hot paths. -
Synthetic path validation (
ovphysx_frame_view.py:560): The clone expansion assumesself._prims[0]is an env_0 prim. Consider validating this assumption when the first prim doesn't containenv_0.
🔍 Overall
The implementation is solid and production-ready. The suggestions above are minor improvements that don't block merging.
Review generated with assistance from Isaac Lab Review Bot
Greptile SummaryThis PR adds
Confidence Score: 4/5Safe to merge with minor follow-up; no data-loss or crash paths identified. The core Warp kernels and tensor-binding data path are logically sound, and the 4096-env manual test gives confidence in the happy path. The _export_env0_only_stage helper has a comment/code mismatch and missing Sdf.ChangeBlock that add memory and warmup overhead but do not break correctness. The shared-config mutation in tests is a test-hygiene concern, not a runtime risk. source/isaaclab_ovphysx/isaaclab_ovphysx/physics/ovphysx_manager.py deserves a second look for the layer-registry and ChangeBlock issues; source/isaaclab_ovphysx/test/sim/test_views_xform_prim_ovphysx.py for the shared-config mutation. Important Files Changed
Sequence DiagramsequenceDiagram
participant IS as InteractiveScene
participant FV as FrameView (factory)
participant OFV as OvPhysxFrameView
participant OM as OvPhysxManager
participant PX as physx wheel
IS->>IS: "clone_usd=True (all backends)"
IS->>OM: _warmup_and_load()
OM->>OM: _export_env0_only_stage(stage, scene.usda)
Note over OM: Export full stage to disk,<br/>open via FindOrOpen,<br/>delete env_1..N specs,<br/>re-export to same file
OM->>PX: physx.add_usd(scene.usda)
PX->>PX: physx.clone() replicates env_1..N at physics layer
FV->>FV: _get_backend() returns ovphysx
FV->>OFV: __new__() OvPhysxFrameView(prim_path)
OFV->>OFV: find_matching_prims(prim_path)
alt PhysX ready
OFV->>OFV: _initialize_impl(physx)
else deferred
OFV->>OM: register_callback(PHYSICS_READY)
OM-->>OFV: _on_physics_ready() calls _initialize_impl(physx)
end
OFV->>PX: create_tensor_binding(pattern, RIGID_BODY_POSE)
PX-->>OFV: pose_binding num_bodies x 7
OFV->>OFV: get_world_poses()
OFV->>PX: pose_binding.read(_pose_buf)
OFV->>OFV: wp.launch _compute_site_world_transforms
OFV-->>FV: positions and orientations as ProxyArray
|
| env_name_re = re.compile(r"^env_(\d+)$") | ||
| names_to_remove = [ | ||
| child_name | ||
| for child_name in list(envs_spec.nameChildren.keys()) | ||
| if (match := env_name_re.match(child_name)) and match.group(1) != "0" | ||
| ] | ||
| for child_name in names_to_remove: | ||
| del envs_spec.nameChildren[child_name] |
There was a problem hiding this comment.
Batch prim deletions without
Sdf.ChangeBlock fires N individual change notifications
Each del envs_spec.nameChildren[child_name] fires a separate USD change notification. At 4096 envs this emits ~4095 notifications before the layer is exported. Wrapping the deletion loop in Sdf.ChangeBlock() batches all spec removals into a single notification burst, which can meaningfully reduce warmup time at large env counts.
| env_name_re = re.compile(r"^env_(\d+)$") | |
| names_to_remove = [ | |
| child_name | |
| for child_name in list(envs_spec.nameChildren.keys()) | |
| if (match := env_name_re.match(child_name)) and match.group(1) != "0" | |
| ] | |
| for child_name in names_to_remove: | |
| del envs_spec.nameChildren[child_name] | |
| env_name_re = re.compile(r"^env_(\d+)$") | |
| names_to_remove = [ | |
| child_name | |
| for child_name in list(envs_spec.nameChildren.keys()) | |
| if (match := env_name_re.match(child_name)) and match.group(1) != "0" | |
| ] | |
| with Sdf.ChangeBlock(): | |
| for child_name in names_to_remove: | |
| del envs_spec.nameChildren[child_name] |
| - **OVPhysX**: :class:`~isaaclab_ovphysx.sim.views.OvPhysxFrameView` | ||
| (Warp-native, reads ``body_q`` via the OVPhysX scene data provider). |
There was a problem hiding this comment.
Docstring says "scene data provider" but the implementation uses a tensor binding
OvPhysxFrameView reads body poses via physx.create_tensor_binding(…, tensor_type=RIGID_BODY_POSE), not through the OVPhysX scene data provider (SDP). The current text implies an SDP dependency that would make users think requires_newton_model=True is needed, which the implementation explicitly avoids.
| - **OVPhysX**: :class:`~isaaclab_ovphysx.sim.views.OvPhysxFrameView` | |
| (Warp-native, reads ``body_q`` via the OVPhysX scene data provider). | |
| - **OVPhysX**: :class:`~isaaclab_ovphysx.sim.views.OvPhysxFrameView` | |
| (Warp-native, reads body poses via an OVPhysX ``RIGID_BODY_POSE`` tensor binding). |
| body_q = self._current_body_q() | ||
|
|
||
| if positions is None or orientations is None: | ||
| cur_pos_ta, cur_quat_ta = self.get_world_poses(indices) | ||
| if positions is None: | ||
| positions = cur_pos_ta.warp | ||
| if orientations is None: | ||
| orientations = cur_quat_ta.warp |
There was a problem hiding this comment.
Double
_current_body_q() read on partial-update path obscures intent
When only one of positions/orientations is None, _current_body_q() is called at line 684 and again inside get_world_poses() at line 687. Because _current_body_q() writes the physics data in-place into _pose_buf and then returns a view of it, the local body_q variable from line 684 silently reflects the second read's data. The kernel at line 697 therefore always uses the second read's body_q, making the first call at line 684 redundant and misleading. The same pattern repeats in set_local_poses.
Add the ovphysx entry to the FrameView factory and teach _get_backend to recognise OvPhysxManager so calls to FrameView(...) return the OVPhysX-backed view instead of falling through to FabricFrameView.
Implement a Warp-native batched-prim view that mirrors NewtonSiteFrameView's site-resolution approach against the OVPhysX scene data provider's body_q array. Each prim resolves at init to a (body_idx, site_local) pair via USD ancestor walk; world poses are computed on GPU as body_q[bid] * site_local. Scales and visibility delegate to an internal UsdFrameView. The view defers initialization to PhysicsEvent.PHYSICS_READY when the SDP is not yet built. Adds contract-suite coverage via the shared frame_view_contract_utils plus OVPhysX-specific tests for factory dispatch, deferred-init guarding, and the requires_newton_model error path.
Read body poses directly from an OVPhysX RIGID_BODY_POSE tensor binding
(mirroring the ContactSensor data path) instead of going through the
scene data provider's Newton state. Scenes that don't declare
requires_newton_model=True (e.g. headless training without a Newton-style
visualizer) can now use OvPhysxFrameView -- the previous design raised at
PHYSICS_READY because the SDP exposes body_q only when the Newton model
build is requested.
Site discovery handles both InteractiveScene modes:
* clone_usd=True: every env has USD prims, one per env matches the
pattern and resolves directly to its rigid-body ancestor.
* clone_usd=False: only env_0 has authored USD prims; env_1..N are
physics-layer clones. The binding row count becomes the authoritative
site count, and per-env site paths are synthesized from the env_0
template prim with env_0 replaced by each row's env_id.
The Warp kernels are unchanged. count, prim_paths, and prims honor the
clone_usd=False expansion. Tests updated to drive _pose_buf directly
(detach the binding after one warm-up read) instead of mutating SDP
body_q, and the now-obsolete requires_newton_model error test is removed.
Under OVPhysX's clone_usd=False scenes (the default for InteractiveScene), USD only holds env_0 -- env_1..N are physics-layer clones via physx.clone(). SensorBase.__init__ derives _num_envs from len(find_matching_prims(...)) so any FrameView-using sensor (RayCaster, MultiMeshRayCaster, Camera) sees _num_envs=1 even when the scene has many envs. The sensor's _reset_mask_torch is then sized 1, and the first reset(env_ids=[0..N-1]) triggers a CUDA assert from out-of-bounds indexing. Walk the call stack at construction time to capture the sensor that owns this view, then at the end of _initialize_impl re-allocate its env-sized buffers (_ALL_ENV_MASK, _reset_mask + torch view, _is_outdated, _timestamp, _timestamp_last_update) to match the OVPhysX RIGID_BODY_POSE binding's row count. Duck-typed -- works for any SensorBase subclass without an isaaclab.sensors import dependency. Mirrors the local fix the OVPhysX ContactSensor already applies to itself at contact_sensor.py:240-248. This is a hack confined to the OVPhysX backend. A cleaner long-term fix would source _num_envs from InteractiveScene.num_envs (or move the under-cloning behaviour to be opt-in for rendering rather than backend- keyed), but both are larger changes that touch core IsaacLab. Verified: Anymal-D rough velocity env with presets=ovphysx + 64 envs now initializes past env.reset() without the CUDA assert.
InteractiveScene now USD-replicates the per-env asset subtree for every backend including OVPhysX (clone_usd=True). OVPhysX's add_usd + physx.clone tolerates the extra USD content at runtime, so the per-env USD prims layer cleanly on top of the physics-side replicas. Sensors that discover _num_envs via find_matching_prims now see N prims under OVPhysX too, matching the behaviour they already had under PhysX and Newton. Removes the OvPhysxFrameView workaround that the previous clone_usd=False behaviour had forced: * OvPhysxFrameView no longer walks the call stack to patch the caller sensor's _num_envs. _num_envs now reaches the binding row count naturally. FrameView parity fixes inspired by a PhysX/Newton review: * OvPhysxFrameView._initialize_impl rejects prim_paths that resolve to a rigid body with a clear ValueError, mirroring the NewtonSiteFrameView guard. FrameView is for non-physics sensor frames; callers should use RigidObject/Articulation for body-level control. * OvPhysxFrameView.get_scales / set_scales docstrings now make explicit that these read/write the static USD xformOp:scale attribute and do not propagate to PhysX collision-shape scale (unlike Newton's shape_scale path).
InteractiveScene now USD-replicates every env's asset subtree for all backends (clone_usd=True from the previous commit), but OvPhysxManager was still handing the full N-env stage to physx.add_usd. The wheel loaded all N USD-defined bodies as independent prims, so the subsequent physx.clone() ran onto already-populated targets and never produced the clone-lineage that the wheel's create_tensor_binding fast path expects. At 4096 envs this turned every binding-creation call into a multi-second USD enumeration -- the hang in articulation init. Re-shape _warmup_and_load to export an env_0-scoped USD file: 1. Export the full stage to disk (existing flatten-and-write). 2. Re-open the exported file as an Sdf.Layer. 3. Delete every /World/envs/env_<i> prim spec for i != 0 from the layer. 4. Re-export. The live USD stage held by SimulationContext is untouched -- sensors (RayCaster, Camera, ContactSensor discovery) still see N envs and discover _num_envs = N correctly. Only the file passed to the wheel is scoped to env_0 + globals. physx.clone() then repopulates env_1..N at the physics layer with proper lineage, and create_tensor_binding walks a 1-USD-path result that auto-extends across the N clones -- the fast path that clone_usd=False used to give us implicitly. Net effect: keeps the previous commit's clone_usd=True flip (sensor parity across backends) while restoring OVPhysX's per-env scaling. No test changes required; the FrameView/ContactSensor suite stays at 27/27 pass on CPU.
Adds a Limitations section to _export_env0_only_stage covering the three assumptions the workaround makes: * Homogeneous envs -- per-env USD-authored physics overrides (mass, friction, collision filters under env_<i!=0>) are dropped from the file handed to physx.add_usd. Sensors and visualizers still see them in the live stage, so a divergence is possible. Per-env state has to be written via the runtime APIs instead. * Global path convention -- physics-relevant prims must live outside /World/envs (or under env_0) to survive the export. * Static topology -- envs added/removed after warmup require a re-warmup with a re-exported stage.
Mirrors the same fix applied to the contact sensor cfg/test; develop's isaaclab.utils no longer re-exports configclass as a callable, so the test's @configclass decorator needs the explicit module import.
83fea49 to
05a330d
Compare
There was a problem hiding this comment.
Code Review: OvPhysxFrameView for OVPhysX Backend
Thank you for this comprehensive PR, @AntoineRichard! The implementation brings full FrameView API parity to the OVPhysX backend with a clean site-based architecture. The USD cloning changes in OvPhysxManager solve a real scaling problem elegantly.
✅ Strengths
- Clean Warp kernel design: The site-based transform computation mirrors
NewtonSiteFrameViewappropriately, with proper handling of world-attached sites viaWORLD_BODY_INDEX = -1 - Deferred initialization: Correctly handles async physics setup via
PHYSICS_READYcallback with clear error messages - Well-documented limitations: The
_export_env0_only_stagedocstring clearly explains the homogeneous-env assumption and its implications - Contract test coverage: Good reuse of
frame_view_contract_utilsfor shared FrameView semantics
📋 Findings
1. Quaternion Convention Clarity (Low Priority)
File: ovphysx_frame_view.py:53-55
The kernel docstring states output is (qx, qy, qz, qw), matching the code:
out_quat[i] = wp.vec4f(q[0], q[1], q[2], q[3]) # q is wp.quatfHowever, wp.quatf storage order is (x, y, z, w) while some Warp APIs expect (w, x, y, z). The code appears correct but consider adding a brief comment confirming the convention matches FabricFrameView and NewtonSiteFrameView for maintainability.
2. USD Layer Export Pattern (Low Priority)
File: ovphysx_manager.py:469-470
del envs_spec.nameChildren[child_name]
...
layer.Export(target_file)The pattern of modifying an Sdf.Layer in-place then re-exporting is correct, but consider calling layer.Clear() or letting the layer go out of scope before the export to ensure the in-memory state does not interfere with subsequent operations in edge cases.
3. Test Coverage: World-Attached Sites (Medium Priority)
File: test_views_xform_prim_ovphysx.py
The test fixture always creates prims under a rigid body (/Cube/CameraMount). Consider adding a test case for prims with no rigid body ancestor to exercise the WORLD_BODY_INDEX == -1 branch:
def test_world_attached_site():
"""Sites without rigid body ancestors use identity body transform."""
# Create an Xform directly under /World (no physics ancestor)
prim = stage.DefinePrim("/World/FreeMarker", "Xform")
view = OvPhysxFrameView("/World/FreeMarker", device=device)
# Verify get_world_poses returns the USD spawn transform4. Rigid Body Rejection Test (Low Priority)
File: ovphysx_frame_view.py:507-513
The guard that rejects prims with RigidBodyAPI is good defensive code:
if self._prim_or_template_has_rigid_body_api(prim):
raise ValueError(...)An explicit test verifying this error path would strengthen coverage.
5. Indexed Query Allocation (Informational)
File: ovphysx_frame_view.py:650-651
pos_buf = wp.zeros(n, dtype=wp.vec3f, device=self._device)
quat_buf = wp.zeros(n, dtype=wp.vec4f, device=self._device)Per-call allocation in indexed queries is fine for correctness. If these become hot paths in tight loops, consider pre-allocating based on max expected index count or documenting the allocation cost.
6. Synthetic Path Assumption (Low Priority)
File: ovphysx_frame_view.py:560
The clone expansion uses self._prims[0] as the template. If the first matched prim happens to not contain env_0 in its path (unlikely but possible with certain regex patterns), the substitution logic would produce incorrect paths. Consider validating that the template prim path contains /env_0/ or logging a warning.
🔍 Architecture Notes
The decision to use a direct RIGID_BODY_POSE tensor binding rather than the scene data provider's Newton model is a good one — it removes the requires_newton_model dependency and makes the view usable in headless training scenarios.
The clone_usd=True change in InteractiveScene with compensating env_0 scoping in OvPhysxManager is a clean solution that maintains sensor discovery parity across backends.
Summary
Solid implementation ready for merge. The suggestions above are minor improvements that do not block approval.
Isaac Lab Review Bot
Update (2cb93be): Reviewed incremental changes:
-
Docstring clarification (
frame_view.py): Updated OvPhysX backend description to reference theRIGID_BODY_POSEtensor binding directly instead of scene data provider — this aligns documentation with the actual implementation. ✅ -
CI compatibility fix (
test_views_xform_prim_ovphysx.py): Addedpytest.importorskip("ovphysx.types")guard with clear reasoning that theisaaclab_ov*CI pattern unintentionally collects these tests. The noqa comments for E402 import-order are appropriate since imports must follow the skip check. This is a pragmatic fix that unblocks CI without masking real issues. ✅
Both changes are minor housekeeping — no functional changes to review. Previous findings remain applicable.
Two PR-review follow-ups for the FrameView change:
* Move ``pytest.importorskip("ovphysx.types", ...)`` above the
``from isaaclab_ovphysx.physics import OvPhysxCfg`` import in the
FrameView test, matching the precedent in
``test_articulation_helpers.py``. Without the guard above, the CI
``isaaclab_ov*`` collection runs ``ModuleNotFoundError`` on Docker
images that don't ship the ovphysx wheel and the required check
fails.
* Fix the :class:`FrameView` docstring: ``OvPhysxFrameView`` reads body
poses via an OVPhysX ``RIGID_BODY_POSE`` tensor binding, not through
the scene data provider. The old text suggested an SDP dependency
that the implementation explicitly avoids.
Description
Adds
OvPhysxFrameView, the OVPhysX-backend implementation of:class:
isaaclab.sim.views.FrameView. The factory inisaaclab.sim.views.frame_viewnow dispatches to it when the activebackend is
"ovphysx", mirroring theFabricFrameView(PhysX) andNewtonSiteFrameView(Newton) entries.OvPhysxFrameViewreads live body poses via the OVPhysX wheel'screate_tensor_binding(pattern, tensor_type=RIGID_BODY_POSE)API andexposes them as
wp.transformfarrays without going through theSceneDataProvider. Sensors built on top of
FrameView(e.g. RayCaster,the ContactSensor pose-tracking path) can now read body transforms
under OVPhysX with the same API surface they use under PhysX and
Newton.
Two scaling-related changes in
InteractiveSceneandOvPhysxManagermake OVPhysX fan-out behave like the other backends:
InteractiveScenenow USD-replicates the per-env asset subtree forevery backend (
clone_usd=True), including OVPhysX. Thefind_matching_prims-based_num_envsdiscovery used by manysensors now sees N prims under OVPhysX, matching PhysX/Newton.
OvPhysxManagerstripsenv_1..Nfrom the in-memory USD beforehanding it to
physx.add_usd. Without this, the wheel's per-USD-pathenumeration scales O(num_envs * bodies) and hangs at 4k+ envs — even
though physics still uses a single replicated template. The env_0
USD is what the wheel ingests; physics replication happens via
physx.clone, identical to the priorclone_usd=Falsepath. A shortcomment on the helper documents the assumption that the per-env
subtrees are structurally homogeneous.
Tested manually with
Isaac-Velocity-Rough-Anymal-D-v0at 4096 envsunder the OVPhysX preset; iter time matches the prior
clone_usd=Falsebaseline.
Fixes #N/A
Type of change
Screenshots
N/A — backend infrastructure change with no UI surface.
Checklist
pre-commitchecks with./isaaclab.sh --formatsource/<pkg>/changelog.d/for every touched package (do not editCHANGELOG.rstor bumpextension.toml— CI handles that)CONTRIBUTORS.mdor my name already exists there