[Newton] Bump Newton pin to v1.2.0rc2#5523
Conversation
After Newton's add_builder copies proto rows into the merged builder
for each environment, every row's label still references the source
proto path. Newton tracks env identity in *_world int companion
columns; IsaacLab keys data flow off USD prim paths, so each row's
label needs to be rewritten to its per-env destination path.
_rename_builder_labels now walks two kinds of label-bearing columns:
* Built-in label arrays (body, joint, shape, articulation,
constraint_mimic, equality_constraint), each paired with its
matching *_world int column.
* String-typed custom-attribute columns (e.g. mujoco:tendon_label)
paired with a references="world" companion at the same frequency.
Any future solver-registered string column at a frequency that has
a references="world" companion is handled automatically.
The prefix match uses a path-separator boundary
(``src_path.rstrip("/") + "/"``) so a source path that is a string
prefix of another (``/Sources/protoA`` vs ``/Sources/protoAB``) does
not cross-contaminate when both feed the same envs.
The function raises ValueError if the parallel arrays of labels and
worlds have mismatched lengths, instead of silently truncating.
Adds test_rename_builder_labels.py with 10 cases covering both
passes, multi-source prefix-overlap regression, sparse env ids,
multi-frequency, multiple string columns at one frequency, and
empty-values pass-through.
Stacks on top of isaac-sim#5523 (Newton v1.2.0rc2 bump). The Newton 1.2
release also includes upstream newton-physics/newton#2659 which
scopes parse_usd's custom-frequency walk natively, removing the
cross-source MjcTendon contamination IsaacLab previously had to
work around at the framework layer.
After Newton's add_builder copies proto rows into the merged builder
for each environment, every row's label still references the source
proto path. Newton tracks env identity in *_world int companion
columns; IsaacLab keys data flow off USD prim paths, so each row's
label needs to be rewritten to its per-env destination path.
_rename_builder_labels now walks two kinds of label-bearing columns:
* Built-in label arrays (body, joint, shape, articulation,
constraint_mimic, equality_constraint), each paired with its
matching *_world int column.
* String-typed custom-attribute columns (e.g. mujoco:tendon_label)
paired with a references="world" companion at the same frequency.
Any future solver-registered string column at a frequency that has
a references="world" companion is handled automatically.
The prefix match uses a path-separator boundary
(src_path.rstrip("/") + "/") so a source path that is a string
prefix of another (/Sources/protoA vs /Sources/protoAB) does not
cross-contaminate when both feed the same envs.
The function raises ValueError if the parallel arrays of labels and
worlds have mismatched lengths, instead of silently truncating.
Adds test_rename_builder_labels.py with 10 cases covering both
passes, multi-source prefix-overlap regression, sparse env ids,
multi-frequency, multiple string columns at one frequency, and
empty-values pass-through.
Stacks on top of isaac-sim#5523 (Newton v1.2.0rc2 bump). The Newton 1.2
release also includes upstream newton-physics/newton#2659 which
scopes parse_usd's custom-frequency walk natively, removing the
cross-source MjcTendon contamination IsaacLab previously had to
work around at the framework layer.
1c55423 to
f1eadc8
Compare
The Newton 1.2.0rc2 release pulls in two IsaacLab-relevant fixes (newton-physics/newton#2678, newton-physics/newton#2720 for SolverKamino reset under world_mask), the upstream tendon-scoping fix from isaac-sim#2659 ("Scope USD custom-frequency parsing"), and a VRAM-leak fix on example reset (isaac-sim#2710). Pin bumps: * source/isaaclab/setup.py: warp-lang==1.12.0 → >=1.13.0, mujoco==3.6.0 → ==3.8.0, mujoco-warp==3.6.0 → ==3.8.0.1. * source/isaaclab_newton/setup.py: same mujoco / mujoco-warp bumps; Newton pin → v1.2.0rc2. * source/isaaclab_visualizers/setup.py: 3 × Newton pin → v1.2.0rc2. * tools/wheel_builder/res/python_packages.toml: same set of pins, mirrored across the wheel-bundling manifest. Code adapts: * source/isaaclab_newton/isaaclab_newton/physics/newton_manager.py and source/isaaclab_ov/isaaclab_ov/renderers/ovrtx_renderer_kernels.py: wp.math.transform_to_matrix → wp.transform_to_matrix. The wp.math namespace was removed in warp-lang 1.13. Smoke verified: Shadow-Hand-Over MAPPO (4 envs, 1 iter) runs clean on the bumped env — simulation init through CUDA graph capture through one training step + checkpoint save, no errors.
f1eadc8 to
c4c2f3d
Compare
Tighten the warp-lang requirement from >=1.13.0 to ==1.13.0 across isaaclab/setup.py, tools/wheel_builder/res/python_packages.toml, and the three changelog fragments. The Newton 1.2.0rc2 toolchain has been validated against this exact warp-lang version; later warp-lang releases have not been smoke-tested here.
Greptile SummaryThis PR bumps the Newton physics dependency from a commit-SHA pin (
Confidence Score: 4/5Safe to merge; the version bumps are consistent across all manifests and the two warp API call sites are correctly updated. The changes are well-scoped: four manifest files get synchronized version bumps and two kernel files get a straightforward one-line API rename. A repo-wide search found no remaining wp.math. usages in source. The only open question is whether the upstream newton tag v1.2.0rc2 is immutable — if it were ever re-tagged, fresh installs could silently diverge from the commit tested here. source/isaaclab_newton/setup.py and source/isaaclab_visualizers/setup.py, where the Newton pin now references a mutable git tag rather than a commit SHA. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
subgraph Before
A1[warp-lang==1.12.0]
A2[mujoco==3.6.0]
A3[mujoco-warp==3.6.0]
A4[newton @ a27277e]
A5[wp.math.transform_to_matrix]
end
subgraph After
B1[warp-lang==1.13.0]
B2[mujoco==3.8.0]
B3[mujoco-warp==3.8.0.1]
B4[newton @ v1.2.0rc2]
B5[wp.transform_to_matrix]
end
subgraph Files Updated
F1[source/isaaclab/setup.py]
F2[source/isaaclab_newton/setup.py]
F3[source/isaaclab_visualizers/setup.py]
F4[tools/wheel_builder/res/python_packages.toml]
F5[newton_manager.py\novrtx_renderer_kernels.py]
end
A1 -->|bumped| B1
A2 -->|bumped| B2
A3 -->|bumped| B3
A4 -->|re-pinned to tag| B4
A5 -->|wp.math removed in 1.13| B5
B1 --> F1 & F2 & F4
B2 --> F1 & F2 & F4
B3 --> F1 & F2 & F4
B4 --> F2 & F3 & F4
B5 --> F5
Reviews (1): Last reviewed commit: "Pin warp-lang to ==1.13.0" | Re-trigger Greptile |
| "mujoco-warp==3.8.0.1", | ||
| "PyOpenGL-accelerate==3.1.10", | ||
| "newton @ git+https://github.com/newton-physics/newton.git@a27277ed49d6f307b8a1e4c394be7e1d14965a62", | ||
| "newton @ git+https://github.com/newton-physics/newton.git@v1.2.0rc2", |
There was a problem hiding this comment.
Mutable tag replaces immutable commit SHA
The previous pin used commit hash a27277ed49d6f307b8a1e4c394be7e1d14965a62, which is immutable. The new pin v1.2.0rc2 is a git tag — if the tag is ever force-pushed on the upstream repo, CI and fresh installs would silently pull different code without any change to this file. The same applies to the three entries in source/isaaclab_visualizers/setup.py and the wheel-builder TOML. If the RC tag is expected to stay frozen (i.e. no re-tagging) this is fine as-is, but pinning to the resolved commit SHA would give stronger reproducibility guarantees.
kellyguo11
left a comment
There was a problem hiding this comment.
isaac sim MR seems to have also updated newton and mujoco usd dependencies - https://gitlab-master.nvidia.com/omniverse/isaac/omni_isaac_sim/-/commit/43148d5d419b491ef2c4350073ef8d4928e60ef5. show we follow to do the same?
|
It looks like newton-usd-schemas and mujoco-usd-converter is transitive through newton or isaac sim. IsaacLab does not pin those. |
The optional [newton] extra in isaaclab_physx pulls Newton in for the visualizer-side scene representation when the user runs PhysX with OV/Rerun/Viser visualizers. Keep its pin synchronized with the Newton pin in isaaclab_newton and isaaclab_visualizers; the previous commit- SHA pin had drifted off the version used elsewhere in the repo. Tighten the changelog wording on the existing fragments to spell out which pins live where: warp-lang/mujoco/mujoco-warp in isaaclab + the wheel-builder TOML; the Newton pin in isaaclab_newton, isaaclab_visualizers, isaaclab_physx, and the wheel-builder TOML.
The changelog CLI rejects slugs containing dots — version-style '1.2.0rc2' embedded in the slug breaks parsing. Rename to 'rc2-bump' without dots. AGENTS.md ## Changelog says the slug defaults to the branch name with '/' replaced by '-', which on this branch produces a dotted slug; the no-dots constraint lives only in tools/changelog/cli.py validation, not in AGENTS.md. Doc fix to follow.
6d41e08 to
58f193e
Compare
Newton 1.2.0rc2 made shape-BVH construction explicit in the
SensorTiledCamera path: passing a non-None state to update() no longer
auto-builds the BVH, and the underlying RenderContext.render now raises
RuntimeError("build_bvh_shape() must be called before rendering
shapes.") if it was never built. CI's three rendering test files
(test_rendering_cartpole.py, test_rendering_dexsuite_kuka_kitless.py,
test_rendering_shadow_hand_kitless.py) all hit this on the first
newton_warp render call after the bump.
Build the shape BVH once in NewtonWarpRenderer.__init__ via
newton.geometry.build_bvh_shape(model, model.state()) — guarded by
shape_count > 0 and bvh_shapes is None so it stays idempotent across
multiple sensors that share the same Newton model. Refit per frame in
render() via newton.geometry.refit_bvh_shape against the current state,
since env body poses move every step.
Recorded in the isaaclab_newton changelog fragment alongside the
existing wp.math.transform_to_matrix → wp.transform_to_matrix adapt.
58f193e to
f8ab14e
Compare
| ), | ||
| ) | ||
|
|
||
| # Newton ``v1.2.0rc2`` made shape-BVH construction explicit; ``SensorTiledCamera.update`` |
There was a problem hiding this comment.
@StafaH - looks like Jichuan included the bvh update in this PR. could you take a look to see if this is correct?
There was a problem hiding this comment.
This is not correct but works. Ideally we want to now move the bvh build and refit outside of the renderer. In 1.2 there is a deprecation path that builds a bvh and refits it to keep old behaviour, so we can just call the original code and newton should handle building if it does not exist.
Newton 1.2.0rc2 requires warp-lang>=1.13, which dropped the warp.torch submodule in favour of top-level warp.* (e.g. wp.torch.device_from_torch became wp.device_from_torch). cuRobo (NVlabs/curobo) — pinned in docker/Dockerfile.curobo and used by isaaclab_mimic — still calls wp.torch.device_from_torch at MotionGenConfig.load_from_robot_config time and raises AttributeError on import. Reconstruct warp.torch as a thin forwarding module at isaaclab_mimic import time so cuRobo's call sites keep working without an upstream patch. Python evaluates isaaclab_mimic/__init__.py before any submodule, including isaaclab_mimic.motion_planners.curobo.curobo_planner, so the shim is in place before cuRobo runs. Idempotent: skipped if warp ever re-introduces wp.torch or cuRobo migrates. Tracking upstream at https://github.com/NVlabs/curobo — once the cuRobo pin in docker/Dockerfile.curobo bumps to a commit that uses the top-level wp.* API directly, drop this shim. The TODO in source/isaaclab_mimic/isaaclab_mimic/__init__.py spells out the removal trigger.
#5538) ## Summary Two unrelated CI breakages on develop, bundled here so develop turns green in one PR. ### 1. Skip the failing viewergl test `test_cartpole_newton_visualizer_viewergl_rgb_motion[physx,newton]` started returning all-black frames on develop after `nvcr.io/nvidian/isaac-sim:latest-develop` flipped to a Kit 110.1.1 + USD 25.11 base. The failure has been deterministic across multiple PRs (#5523, #5495, #5408, …). Investigation so far has ruled out: - PR #5521 (revert in #5539 still failed) - Newton 1.0 → 1.2.0rc2 viewer code regression (only 7-line addition; ViewerGL alone yields 1.08M nonzero pixels) - warp 1.12 → 1.13 RegisteredGLBuffer ABI (byte-identical) - Module-load side effects of `isaaclab_physx.renderers` - CUDA-GL interop (PR #5540 diagnostic confirms direct CPU FBO readback also returns zeros, with `GL_NO_ERROR`) - GL context-currency (PR #5541 H6 attempt: still fails) - GL/CUDA sync (PR #5542 H4 attempt: still fails) Diagnostic output (PR #5540 v2): ``` [VIZDIAG] fbo=c_uint(8) pbo=None size=600x600 [VIZDIAG] glGetError before: GL_NO_ERROR [VIZDIAG] CPU-readback: nonzero=0/1080000 max=0 err=GL_NO_ERROR [VIZDIAG] PBO-result: nonzero=0/1080000 max=0 ``` The FBO itself is empty — Newton's pyglet/EGL renderer is not depositing pixels under Kit 110.1.1, even though `tiled_camera_rgb_non_black` (Kit RTX path) on the same env passes. Underlying root cause still being chased; this PR ships the skip to unblock develop. ### 2. Fix warp intersphinx 404 in docs build `https://nvidia.github.io/warp/objects.inv` started returning 404 — Warp's `objects.inv` only lives at `/stable/` and `/latest/` now. With Sphinx's `warnings_treated_as_errors`, the broken intersphinx fetch fails the docs build on every PR. Pinning to `/stable/` (matches the existing PyTorch `/docs/2.11/` workaround pattern in the same file). Verified `https://nvidia.github.io/warp/stable/objects.inv` returns 200. ## Test plan - [x] CI `isaaclab_visualizers` on this branch — was passing earlier with the skip; will re-verify with the bundled docs fix - [ ] CI `Build Latest Docs` on this branch — must turn green (was failing on every recent PR before this fix) ## Re-enable plan Once the underlying viewergl bug is identified and fixed, drop the `@pytest.mark.skip` decorator and remove the `jichuanh-disable-viewergl-flaky.skip` fragment.
After Newton's add_builder copies proto rows into the merged builder
for each environment, every row's label still references the source
proto path. Newton tracks env identity in *_world int companion
columns; IsaacLab keys data flow off USD prim paths, so each row's
label needs to be rewritten to its per-env destination path.
_rename_builder_labels now walks two kinds of label-bearing columns:
* Built-in label arrays (body, joint, shape, articulation,
constraint_mimic, equality_constraint), each paired with its
matching *_world int column.
* String-typed custom-attribute columns (e.g. mujoco:tendon_label)
paired with a references="world" companion at the same frequency.
Any future solver-registered string column at a frequency that has
a references="world" companion is handled automatically.
The prefix match uses a path-separator boundary
(src_path.rstrip("/") + "/") so a source path that is a string
prefix of another (/Sources/protoA vs /Sources/protoAB) does not
cross-contaminate when both feed the same envs.
The function raises ValueError if the parallel arrays of labels and
worlds have mismatched lengths, instead of silently truncating.
Adds test_rename_builder_labels.py with 10 cases covering both
passes, multi-source prefix-overlap regression, sparse env ids,
multi-frequency, multiple string columns at one frequency, and
empty-values pass-through.
Stacks on top of isaac-sim#5523 (Newton v1.2.0rc2 bump). The Newton 1.2
release also includes upstream newton-physics/newton#2659 which
scopes parse_usd's custom-frequency walk natively, removing the
cross-source MjcTendon contamination IsaacLab previously had to
work around at the framework layer.
Summary
Bumps the Newton pin to
v1.2.0rc2, which pulls in IsaacLab-relevant fixes plus the upstream tendon-scoping fix.What's new in Newton v1.2.0rc2 vs IsaacLab's current pin (
a27277e)The current IsaacLab Newton pin is from late April; v1.2.0rc2 is the latest release-candidate cut. Notable fixes pulled in:
parse_usdnow scopes the custom-frequency walk toroot_pathnatively.SolverKaminoreset underworld_mask.Required dep bumps
Newton 1.2.0rc2's `pyproject.toml` requires:
Pins updated in:
Code adapts
`warp-lang` 1.13 removed the `wp.math` namespace. Two IsaacLab call sites use it:
Both rewritten as `wp.math.transform_to_matrix(...)` → `wp.transform_to_matrix(...)`. That's the only IsaacLab-side adapt needed.
Test plan
Caveat
Smoke covered Shadow-Hand-Over MAPPO. Other envs with different sensors / renderers / collision setups could surface warp 1.13 or mujoco 3.8 differences the smoke didn't exercise; full PR CI catches them.