[Newton] Implements Frame Transformer Sensor#4822
[Newton] Implements Frame Transformer Sensor#4822camevor merged 12 commits intoisaac-sim:developfrom
Conversation
camevor
force-pushed
the
ca/newton-frame-transformer
branch
from
d01bb4f to
3be332e
Compare
camevor
changed the title
Greptile SummaryImplements the Newton
Confidence Score: 4/5
Important Files Changed
Sequence DiagramsequenceDiagram
participant FT as FrameTransformer
participant NM as NewtonManager
participant NR as newton_replicate
participant SFT as SensorFrameTransform
Note over FT,NM: __init__ phase (before simulation)
FT->>NM: cl_register_site(None, identity) → world_origin_label
FT->>NM: cl_register_site(prim_path, source_offset) → source_label
FT->>NM: cl_register_site(target_path, target_offset) → target_label(s)
NM-->>NM: Store in _cl_pending_sites
Note over NM,NR: Replication path (newton_replicate)
NR->>NM: _cl_inject_sites(builder, protos)
NM-->>NR: global_sites, proto_sites
NR->>NR: Compute per-world shape offsets
NR->>NM: _cl_site_index_map = {...}
Note over NM: OR Fallback path (start_simulation)
NM->>NM: _cl_inject_sites_fallback()
NM-->>NM: Populate _cl_site_index_map
Note over FT,SFT: PHYSICS_READY callback
FT->>NM: Read _cl_site_index_map
FT->>FT: _validate_site_map()
FT->>FT: _build_sensor_index_lists()
FT->>NM: add_frame_transform_sensor(shapes, refs)
NM->>SFT: Create SensorFrameTransform
NM-->>FT: sensor_index
Note over FT,SFT: Update loop
NM->>SFT: sensor.update(state)
FT->>FT: copy_from_newton_kernel
FT->>FT: compose_target_world_kernel
Last reviewed commit: a36ead3 |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
camevor
force-pushed
the
ca/newton-frame-transformer
branch
from
61d44e7 to
b7f551a
Compare
![isaaclab-review-bot[bot]](https://avatars.githubusercontent.com/u/157846462?s=60&v=4){kind=small}
There was a problem hiding this comment.
Review: Newton Frame Transformer Sensor
This is a substantial rewrite that replaces the old per-view body lookup approach with a site-injection pipeline backed by Newton's SensorFrameTransform. The architecture is cleaner and the test coverage is thorough. A few items worth addressing:
Architecture
Strengths:
- The site-injection pipeline (
cl_register_site→_cl_inject_sites→ replication) is well-designed. Registering sites before replication so they automatically replicate per-world is the right approach. - Zero-copy
transform_to_vec_quatutility is clean and well-bounded (1D/2D/3D). - The
_validate_site_mapand_build_sensor_index_listsstatic methods are nicely testable in isolation (and indeed tested intest_site_injection.py).
Issues
-
Mutable class-level state on
NewtonManager—_cl_pending_sites,_cl_site_index_map,_newton_frame_transform_sensorsare all mutable dicts/lists as class attributes. This is a pre-existing pattern in the codebase but adding more of it increases the risk of stale state between scene reloads. Theclear()method resets them, but any new path that skipsclear()will inherit ghost sites from a previous scene. Consider adding a defensive check at the top ofcl_register_siteor_cl_inject_sitesif the model is already finalized. -
compose_target_world_kernelorder — The kernel computessource_world * target_relative. In rigid-body math, iftarget_relativeis the transform of the target in the source frame, then the world transform should besource_world ∘ target_relative, which is whatwp.transform_multiply(a, b)computes (a * b). This looks correct, but the docstrings in the data class say "target-relative transforms" without explicitly specifying the convention. Worth a one-line comment confirming:T_world_target = T_world_source * T_source_target. -
Quaternion convention in docstrings — The base class specifies
(x, y, z, w)order for quaternions and Warp'squatfis(x, y, z, w). The Newton data class docstrings say(xyzw)which is consistent — good. However, the old code had explicitsplit_transform_to_quat4labconverting to(w, x, y, z). The new implementation usestransform_to_vec_quatwhich returns rawwp.quatfviews (xyzw). This is correct for the Warp-native interface, but if any downstream code expects(w, x, y, z)fromsource_quat_wortarget_quat_w, this would be a breaking behavioral change for Newton users who relied on the old vec4f (wxyz) format. The tests usewp.to_torch()and compare against articulation data which is also xyzw, so the tests pass, but it's worth flagging this as a deliberate convention change in the changelog. -
resolve_matching_namesraise_when_no_match=Falsereturns([], [])on partial match — If 2 of 3 patterns match, the current implementation returns empty lists rather than the 2 that did match. This is fine for the site injection use case (you want all-or-nothing matching per prototype), but the behavior is a bit surprising for a general-purpose utility. The docstring should clarify this returns empty on any unmatched pattern, not just when all fail. -
_cl_inject_sitesclears_cl_pending_sites— Both_cl_inject_sites(replication path) and_cl_inject_sites_fallback(non-replication path) clear pending sites. If for some reason both are called (e.g., a code path bug), the second call would silently do nothing. This is probably fine but adding a debug log when pending is already empty would help future debugging. -
shape_labelsaccess pattern in_build_sensor_index_lists— The method accessesshape_labels[site_idx]for wildcard expansion. This assumesshape_labelsis indexable by int (list) or has int keys (dict). The type hint sayslist[str] | dict[int, str]. Thebuilder.shape_labelin Newton is typically a list, so this works, butshape_labels[site_idx]could IndexError if the site index exceeds the label list length (e.g., if sites were added after the labels snapshot). Worth a bounds check ortry/exceptwith a clear error message.
Nits
frame_transformer_data.py: TheFrameTransformerData.__init__sets_target_frame_names = Nonebuttarget_frame_namesproperty accesses it unconditionally. Ifdata.target_frame_namesis accessed before_initialize_impl, it returnsNonerather than raising — consider making it raise explicitly._default_root_state = Noneaddition inrigid_object_data.py— this looks like an unrelated fix. Should it be in a separate PR or at least mentioned in the PR description?interactive_scene.pyremoves a comment block about sensor template paths. If the logic is still correct, that's fine, but the comment was useful context.
Tests
Excellent coverage: feet w.r.t. base, feet w.r.t. thigh, robot-to-external-cube, offset frames, all-bodies wildcard, duplicate body names, and isolated unit tests for site injection/validation. The parametrized duplicate-body-name test with source_robot × path_prefix is particularly good.
Overall this is solid work. The site injection pipeline is the right abstraction for Newton's model-building approach.
|
@isaaclab-review-bot
Nits:
|
Assertions are used in this PR to verify internal invariants, failures of which are unrecoverable. User input and configuration validation does not use assertions but raises errors instead. |
|
That's a valid distinction. Using assertions for internal invariants (which indicate programming errors) while raising explicit errors for user-facing validation is a common and acceptable pattern. Since these are checking preconditions that should never fail in correct usage, the risk of Tip: You can give me more context with .greptile/rules so I get this right next time. |
AntoineRichard
left a comment
AntoineRichard
left a comment
There was a problem hiding this comment.
It looks ok, it's quite a substantial change with the site injection, the frame transformer does gets a nice update. @ooctipus I think some of the changes in this PR will require to take a fresh look at the cloner when you get back.
| # Sensor may be nested under a proto_asset_N prim (e.g. a camera on a robot | ||
| # link). Search for the actual template location so spawning succeeds even | ||
| # though the parent asset lives at template_root/<Asset>/proto_asset_0/... |
There was a problem hiding this comment.
Is this no longer true?
There was a problem hiding this comment.
Good catch, thanks. Restored it.
| # -- default root pose and velocity | ||
| self._default_root_pose = wp.zeros((self._num_instances,), dtype=wp.transformf, device=self.device) | ||
| self._default_root_vel = wp.zeros((self._num_instances,), dtype=wp.spatial_vectorf, device=self.device) | ||
| self._default_root_state = None # lazily allocated by deprecated default_root_state property |
There was a problem hiding this comment.
Was that missing var not caught by the tests?
There was a problem hiding this comment.
Good question, I'm not sure why it wasn't caught.
| if not env_mask[env]: | ||
| return | ||
|
|
||
| target_transforms_w[env, tgt] = wp.transform_multiply(source_transforms[env], target_transforms[env, tgt]) |
There was a problem hiding this comment.
Is that better than a plain source_transforms[env] * target_transforms[env, tgt]
There was a problem hiding this comment.
Good point, this is exactly the same.
| def transform_to_vec_quat( | ||
| t: wp.array, | ||
| ) -> tuple[wp.array, wp.array]: | ||
| """Split a wp.transformf array into position (vec3f) and quaternion (quatf) arrays. | ||
|
|
||
| Zero-copy: returns views into the same underlying memory. | ||
|
|
||
| Args: | ||
| t: Array of transforms (dtype=wp.transformf). Shape ``(N,)``, ``(N, M)``, or ``(N, M, K)``. | ||
|
|
||
| Returns: | ||
| Tuple of (positions, quaternions) as warp array views with matching dimensionality. | ||
|
|
||
| Raises: | ||
| TypeError: If *t* does not have dtype ``wp.transformf``. | ||
| """ | ||
| if t.dtype != wp.transformf: | ||
| raise TypeError(f"Expected wp.transformf array, got dtype={t.dtype}") | ||
| floats = t.view(wp.float32) | ||
| if t.ndim == 1: | ||
| return floats[:, :3].view(wp.vec3f), floats[:, 3:].view(wp.quatf) | ||
| if t.ndim == 2: | ||
| return floats[:, :, :3].view(wp.vec3f), floats[:, :, 3:].view(wp.quatf) | ||
| if t.ndim == 3: | ||
| return floats[:, :, :, :3].view(wp.vec3f), floats[:, :, :, 3:].view(wp.quatf) | ||
| raise ValueError(f"Expected 1D, 2D, or 3D transform array, got ndim={t.ndim}") |
There was a problem hiding this comment.
We can do slicing now? Might be interesting to keep this in mind when we'll review the asset code. There we use some quite violent code to generate this sort of things.
There was a problem hiding this comment.
Agreed, this could simplify code in a few places.
There was a problem hiding this comment.
Do we have a quick benchmark on the initialization time of this sensor? (To avoid a contact sensor init situation)
There was a problem hiding this comment.
Good point, I'll check.
There was a problem hiding this comment.
A brief checks on 128 and 2048 worlds looks good:
Setup: NVIDIA L40, Python 3.11, Newton 1.1.0.dev0, kitless. Scene: ANYmal C + cube + FrameTransformer (4 feet targets). 200 steps, 20 warmup, 5 runs each.
Initialization (ms, first run discarded)
| Phase | develop 128 | this PR 128 | develop 2048 | this PR 2048 |
|---|---|---|---|---|
scene_construction |
5237 ±783 | 3878 ±415 | 27285 ±29592 | 23722 ±8969 |
sim_reset |
3805 ±143 | 3650 ±65 | 8480 ±521 | 7666 ±176 |
| total | 9042 ±896 | 7528 ±394 | 35764 ±29403 | 31388 ±9018 |
scene_construction has very large run-to-run variance. It dominates the init measurement and makes the two branches indistinguishable on init time. sim_reset is more stable and shows this PR ~1s faster at 2048 (9036 vs 8005), likely from the slimmer _initialize_impl.
No init regression from the new implementation.
Step Breakdown (ms, median, first run discarded)
| Phase | develop 128 | this PR 128 | develop 2048 | this PR 2048 |
|---|---|---|---|---|
write_data_to_sim |
1.042 | 1.042 | 1.237 | 1.229 |
sim.step |
0.781 | 0.792 | 1.188 | 1.169 |
scene.update |
0.180 | 0.166 | 1.861 | 0.160 |
| total | 2.003 | 1.999 | 4.289 | 2.560 |
Step-time std across 5 runs is ≤0.03 ms for all phases.
Step time scales much better with this implementation.
There was a problem hiding this comment.
Thanks for commenting the mods to the cloner here. @kellyguo11 we need to start considering another refactor of this class. I think after milestone 2 we need to look into this. That class is getting out of hand.
There was a problem hiding this comment.
Follow-up Review (3 commits since b7f551a → 61c83b1)
Changes Since Last Bot Review
- "Simplify transform composition" —
wp.transform_multiply(a, b)→a * bincompose_target_world_kernel. Addressed AntoineRichard's inline comment. ✅ - "Restore deleted comment" — Comment block in
interactive_scene.pyrestored per AntoineRichard's catch. ✅ - "Tighten type hint" — Minor typing improvement. ✅
- Merge of develop — No conflicts with PR files.
Previous Review Items — Status
From the initial bot review (4077361704):
| # | Item | Status |
|---|---|---|
| 1 | Mutable class-level state on NewtonManager | Acknowledged as pre-existing pattern. clear() resets all new attributes. |
| 2 | compose_target_world_kernel convention comment |
Simplified to * operator. Convention is clear from context (T_w_target = T_w_source * T_source_target). |
| 3 | Quaternion convention (xyzw vs wxyz) | Tests verify xyzw throughout. Deliberate convention change from old split_transform_to_quat4lab (wxyz). Not called out in changelog — see below. |
| 4 | resolve_matching_names partial-match behavior |
Unchanged. Docstring updated. |
| 5 | Double-clear of _cl_pending_sites |
Acknowledged. Both paths clear, second is benign no-op. |
| 6 | shape_labels bounds check |
Unchanged. |
Remaining Items Worth Noting
Quaternion convention change should be documented in the changelog. The old Newton FrameTransformerData returned wp.vec4f in (w,x,y,z) order from split_transform_to_quat4lab. The new implementation returns wp.quatf views in (x,y,z,w) order (Warp native). Any downstream code that indexed source_quat_w or target_quat_w expecting [0]=w will silently get wrong values. The tests pass because they compare against articulation data which is also xyzw, but existing user code migrating from the old Newton FrameTransformer would break. A one-liner in the changelog "Changed" section noting this convention shift would save debugging time.
FrameTransformerData.__init__ sets _target_frame_names = None but the target_frame_names property returns it unconditionally. Accessing data.target_frame_names before _initialize_impl returns None instead of raising. Minor, but an explicit guard would be cleaner.
Newton changelog date mismatch: 0.5.11 (2026-04-07) but the actual commits are from 2026-04-08 through 2026-04-10. Minor nit.
Overall
All reviewer feedback from AntoineRichard has been addressed. The architecture is clean, test coverage is excellent (813 lines of integration tests + 320 lines of unit tests), and the benchmark numbers camevor posted show a 42% step-time improvement at 2048 envs with comparable init time. The site injection pipeline is the right approach for Newton's builder-based replication model.
No blocking issues. The quaternion convention documentation is the most important remaining item — it's a behavioral change that could bite migrating users.
There was a problem hiding this comment.
🤖 Isaac Lab Review Bot
Summary
This PR rewrites the Newton FrameTransformer sensor from an inline per-view body lookup + Warp kernel approach to a clean site-injection pipeline backed by SensorFrameTransform. The design is solid — registering sites before replication means they automatically replicate per-world without post-hoc index patching. The implementation is well-structured with good separation of concerns (site registration → injection → validation → sensor creation), and ships with thorough test coverage (1100+ lines across integration and unit tests).
Design Assessment
Design is sound. The site-injection approach via NewtonManager.cl_register_site is a clean pattern that lets sensors declare their needs at __init__ time and have the replication machinery handle per-world duplication transparently. The zero-copy transform_to_vec_quat views are an elegant way to expose position/quaternion components without buffer duplication. The strided flat sensor layout ([source₀, target₀₀..target₀ₘ, source₁, target₁₀..target₁ₘ, ...]) is efficient for GPU reads.
The approach correctly handles three code paths:
- Replication path (
newton_replicate) — sites injected into prototypes beforeadd_builder - USD stage path (
instantiate_builder_from_stage) — sites injected into proto, then replicated per-env - Fallback path (
_cl_inject_sites_fallback) — sites injected into flat builder (no replication)
Findings
🟡 Warning: assert used for runtime validation — stripped by python -O — source/isaaclab_newton/isaaclab_newton/sensors/frame_transformer/frame_transformer.py:129,201,222
Three assert statements are used to validate that site labels exist in the site map during _initialize_impl and _validate_site_map. These checks guard against real misconfiguration (e.g., a sensor registering a site that wasn't injected) but would silently disappear if Python is run with optimizations enabled (-O or -OO), leading to confusing KeyError or TypeError further downstream instead of a clear diagnostic message.
The PhysX counterpart doesn't use assert for this kind of validation.
# Line 129 — replace:
# assert self._world_origin_label in site_map
# with:
if self._world_origin_label not in site_map:
raise RuntimeError(
f"World-origin site '{self._world_origin_label}' not found in site map. "
"Ensure NewtonManager site injection ran before initialization."
)
# Lines 201-205 — replace assert with:
if source_label not in site_map:
raise ValueError(
f"FrameTransformer source '{source_prim_path}' (site label '{source_label}') "
"not found in NewtonManager._cl_site_index_map."
)
# Lines 222-225 — replace assert with:
if label not in site_map:
raise ValueError(
f"FrameTransformer target '{target_prim_paths[tgt_idx]}' (site label '{label}') "
"not found in NewtonManager._cl_site_index_map."
)
🟡 Warning: No length validation in add_frame_transform_sensor — source/isaaclab_newton/isaaclab_newton/physics/newton_manager.py:1155
add_frame_transform_sensor passes shapes and reference_sites directly to SensorFrameTransform without validating that they have the same length. A mismatch would likely cause a confusing error inside Newton rather than a clear diagnostic at the Isaac Lab level.
# Suggested guard at the top of add_frame_transform_sensor:
if len(shapes) != len(reference_sites):
raise ValueError(
f"shapes and reference_sites must have the same length, "
f"got {len(shapes)} and {len(reference_sites)}"
)🔵 Suggestion: _build_sensor_index_lists accesses shape_labels by integer index — source/isaaclab_newton/isaaclab_newton/sensors/frame_transformer/frame_transformer.py:271
In the wildcard expansion path, shape_labels[site_idx] indexes builder.shape_label using the site's shape index. This works correctly because add_site returns a valid shape index, but accessing a list by an external integer without bounds checking means a bug in site injection could produce an opaque IndexError. Consider adding a guard or a comment documenting the invariant:
if n_bodies > 1:
site_idx = per_world[0][k]
# site_idx is guaranteed valid by _cl_inject_sites / add_site
expanded_names.append(shape_labels[site_idx].rsplit("/", 2)[-2])Test Coverage
Coverage is excellent for a new feature PR:
- Integration tests (
test_frame_transformer.py, 813 lines): Covers feet-wrt-base, feet-wrt-thigh, robot-to-external-cube, offset frames, wildcard all-bodies matching, duplicate body names across robots, and sensor print. Tests run with 2 envs, include periodic resets, and verify both world-frame and source-relative transforms against articulation ground truth. - Unit tests (
test_site_injection.py, 320 lines): Coverstransform_to_vec_quat(1D/2D split, zero-copy, dtype errors), fallback site injection (global, local, wildcard, no-match),_validate_site_map(source/target validation, edge cases), and_build_sensor_index_lists(zero targets, single target, multi-env, wildcard expansion). - Gap: No explicit test for the
_cl_inject_sitesreplication path (only the fallback path is unit-tested), though the integration tests exercise it indirectly viaInteractiveScenewithnum_envs=2.
CI Status
- pre-commit: ✅ pass
- license-check: ⏳ pending
- Build Latest Docs: ⏳ pending
- Check for Broken Links: ✅ pass
Verdict
COMMENT
Well-structured rewrite with strong test coverage. The two warnings above are minor hardening improvements — the assert→raise change is the most actionable since it affects error reporting in optimized Python runtimes. Overall this is clean, well-tested code ready for merge once the CI checks complete.
| site_map = NewtonManager._cl_site_index_map | ||
|
|
||
| # Resolve and validate per-env site indices | ||
| assert self._world_origin_label in site_map |
There was a problem hiding this comment.
🟡 Warning: assert is stripped by python -O, turning this into a no-op. Consider:
| assert self._world_origin_label in site_map | |
| if self._world_origin_label not in site_map: | |
| raise RuntimeError( | |
| f"World-origin site '{self._world_origin_label}' not found in site map. " | |
| "Ensure NewtonManager site injection ran before initialization." | |
| ) |
Same applies to the assert statements in _validate_site_map at lines 201 and 222.
| """ | ||
| sensor = SensorFrameTransform( | ||
| cls._model, | ||
| shapes=shapes, |
There was a problem hiding this comment.
🟡 Warning: No validation that shapes and reference_sites have the same length. A mismatch would produce a confusing error inside Newton rather than a clear diagnostic.
| shapes=shapes, | |
| sensor = SensorFrameTransform( | |
| cls._model, | |
| shapes=shapes, | |
| reference_sites=reference_sites, | |
| ) |
Consider adding a guard:
if len(shapes) != len(reference_sites):
raise ValueError(
f"shapes and reference_sites must have the same length, "
f"got {len(shapes)} and {len(reference_sites)}"
)
# Description - Rewrites the Newton `FrameTransformer` sensor to use a site-injection pipeline backed by `SensorFrameTransform`. Source and target sites are registered before replication via `NewtonManager.cl_register_site`, so they automatically replicate per-world. - Replaces the old per-view body lookup and inline Warp kernels with zero-copy `transform_to_vec_quat` views and a flat strided sensor output. - Adds wildcard body matching for target frames (e.g. `"Robot/finger.*"` expands to one target per matched body per env). - Also includes a one-line fix to `RigidObjectData` initializing `_default_root_state = None` for lazy allocation, and a minor contact sensor fix. ## Checklist - [ ] I have read and understood the [contribution guidelines](https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html) - [ ] I have run the [`pre-commit` checks](https://pre-commit.com/) with `./isaaclab.sh --format` - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have updated the changelog and the corresponding version in the extension's `config/extension.toml` file - [ ] I have added my name to the `CONTRIBUTORS.md` or my name already exists there
## Summary Fix incorrect attribute name `contact_margin` → `gap` on Newton `ShapeConfig` in `NewtonManager.create_builder()`. Newton PR #1732 renamed `contact_margin` to `gap` (broad-phase AABB expansion). The IsaacLab code was never updated, creating a dead attribute that Python silently accepted. The intended 1 cm default shape gap was never applied. ## Newton PR #1732 rename mapping | Old field | New field | Semantics | |-----------|-----------|-----------| | `thickness` | `margin` | Shape surface extension (affects contact forces) | | `contact_margin` | **`gap`** | Broad-phase AABB expansion (detection range only) | | `rigid_contact_margin` | `rigid_gap` | Builder-level default gap (already 0.1) | ## Timeline | Date | Newton | IsaacLab | `contact_margin` valid? | |------|--------|----------|------------------------| | Feb 19 | pin: `51ce35e` (has `contact_margin`) | #4646 adds `contact_margin = 0.01` | Yes | | Feb 24 | **PR #1732 renames `contact_margin` → `gap`** | — | — | | Mar 2 | pin updated to `v0.2.3` (after rename) | #4761 keeps `contact_margin` | **No — broken** | | Mar 9 | — | #4883 removes the line | Removed | | Apr 13 | — | #4822 re-adds `contact_margin` | **No — still broken** | ## Ablation: gap vs margin We conducted an ablation study to understand the impact. Note: `margin` (shape surface extension) is a different field from `gap` (broad-phase range). The original code intended to set `gap`, but setting `margin` also has a significant positive effect on training for rough terrain locomotion. | Setting | `gap` | `margin` | Go1 Reward (300 iter) | Effect | |---------|-------|----------|----------------------|--------| | Dead field (baseline) | 0.1 (default) | 0.0 | ~1.0 | — | | `gap=0.01` only | 0.01 | 0.0 | 0.66 | No training improvement | | `margin=0.01` only | 0.1 (default) | 0.01 | 18.77 | Major improvement | | `gap=0.01` + `margin=0.01` | 0.01 | 0.01 | 16.54 | Similar to margin-only | **This PR fixes the correct field migration** (`contact_margin` → `gap`). The `margin` setting for rough terrain contact quality will be addressed separately in the rough terrain env PR (#5248) via a new `default_shape_margin` config field on `NewtonCfg`. ## Test plan - [x] Verified `contact_margin` is not a field on `ShapeConfig` (Newton 1.1.0.dev0) - [x] Verified `gap` is the correct replacement field per Newton PR #1732 - [x] Confirmed by camevor (Newton developer) - [x] Ablation study: gap alone doesn't change training, so existing tests should pass Co-authored-by: Antoine RICHARD <antoiner@nvidia.com>
2 participants
Description
FrameTransformersensor to use a site-injection pipeline backed bySensorFrameTransform. Source and target sites are registered before replication viaNewtonManager.cl_register_site, so they automatically replicate per-world.transform_to_vec_quatviews and a flat strided sensor output."Robot/finger.*"expands to one target per matched body per env).RigidObjectDatainitializing_default_root_state = Nonefor lazy allocation, and a minor contact sensor fix.Checklist
pre-commitchecks with./isaaclab.sh --formatconfig/extension.tomlfileCONTRIBUTORS.mdor my name already exists there