Skip to content

[Newton] Bump Newton pin to v1.2.0rc2#5523

Merged
hujc7 merged 8 commits intoisaac-sim:developfrom
hujc7:jichuanh/newton-1.2.0rc2-bump
May 8, 2026
Merged

[Newton] Bump Newton pin to v1.2.0rc2#5523
hujc7 merged 8 commits intoisaac-sim:developfrom
hujc7:jichuanh/newton-1.2.0rc2-bump

Conversation

@hujc7
Copy link
Copy Markdown
Collaborator

@hujc7 hujc7 commented May 6, 2026

Summary

Bumps the Newton pin to v1.2.0rc2, which pulls in IsaacLab-relevant fixes plus the upstream tendon-scoping fix.

What's new in Newton v1.2.0rc2 vs IsaacLab's current pin (a27277e)

The current IsaacLab Newton pin is from late April; v1.2.0rc2 is the latest release-candidate cut. Notable fixes pulled in:

Required dep bumps

Newton 1.2.0rc2's `pyproject.toml` requires:

  • `warp-lang==1.13.0`
  • `mujoco==3.8.0` (was 3.6.0)
  • `mujoco-warp==3.8.0.1` (was 3.6.0)

Pins updated in:

File Change
`source/isaaclab/setup.py` `warp-lang==1.12.0` → `==1.13.0`; `mujoco==3.6.0` → `==3.8.0`; `mujoco-warp==3.6.0` → `==3.8.0.1`
`source/isaaclab_newton/setup.py` mujoco / mujoco-warp bumps; Newton pin → `v1.2.0rc2`
`source/isaaclab_visualizers/setup.py` 3× Newton pin → `v1.2.0rc2`
`tools/wheel_builder/res/python_packages.toml` All four pins mirrored

Code adapts

`warp-lang` 1.13 removed the `wp.math` namespace. Two IsaacLab call sites use it:

  • `source/isaaclab_newton/isaaclab_newton/physics/newton_manager.py:72`
  • `source/isaaclab_ov/isaaclab_ov/renderers/ovrtx_renderer_kernels.py:330`

Both rewritten as `wp.math.transform_to_matrix(...)` → `wp.transform_to_matrix(...)`. That's the only IsaacLab-side adapt needed.

Test plan

  • `./isaaclab.sh -i newton` clean install against the bumped pins.
  • `pip list` confirms `newton 1.2.0rc2`, `warp-lang 1.13.0`, `mujoco 3.8.0`, `mujoco-warp 3.8.0.1`.
  • Sanity smoke: Shadow-Hand-Over MAPPO (4 envs, 1 iter) runs clean — simulation init through CUDA graph capture through one training step + checkpoint save, no errors.
  • Pre-commit clean.

Caveat

Smoke covered Shadow-Hand-Over MAPPO. Other envs with different sensors / renderers / collision setups could surface warp 1.13 or mujoco 3.8 differences the smoke didn't exercise; full PR CI catches them.

@github-actions github-actions Bot added isaac-lab Related to Isaac Lab team infrastructure labels May 6, 2026
hujc7 added a commit to hujc7/IsaacLab that referenced this pull request May 6, 2026
After Newton's add_builder copies proto rows into the merged builder
for each environment, every row's label still references the source
proto path. Newton tracks env identity in *_world int companion
columns; IsaacLab keys data flow off USD prim paths, so each row's
label needs to be rewritten to its per-env destination path.

_rename_builder_labels now walks two kinds of label-bearing columns:

* Built-in label arrays (body, joint, shape, articulation,
  constraint_mimic, equality_constraint), each paired with its
  matching *_world int column.
* String-typed custom-attribute columns (e.g. mujoco:tendon_label)
  paired with a references="world" companion at the same frequency.
  Any future solver-registered string column at a frequency that has
  a references="world" companion is handled automatically.

The prefix match uses a path-separator boundary
(``src_path.rstrip("/") + "/"``) so a source path that is a string
prefix of another (``/Sources/protoA`` vs ``/Sources/protoAB``) does
not cross-contaminate when both feed the same envs.

The function raises ValueError if the parallel arrays of labels and
worlds have mismatched lengths, instead of silently truncating.

Adds test_rename_builder_labels.py with 10 cases covering both
passes, multi-source prefix-overlap regression, sparse env ids,
multi-frequency, multiple string columns at one frequency, and
empty-values pass-through.

Stacks on top of isaac-sim#5523 (Newton v1.2.0rc2 bump). The Newton 1.2
release also includes upstream newton-physics/newton#2659 which
scopes parse_usd's custom-frequency walk natively, removing the
cross-source MjcTendon contamination IsaacLab previously had to
work around at the framework layer.
hujc7 added a commit to hujc7/IsaacLab that referenced this pull request May 6, 2026
After Newton's add_builder copies proto rows into the merged builder
for each environment, every row's label still references the source
proto path. Newton tracks env identity in *_world int companion
columns; IsaacLab keys data flow off USD prim paths, so each row's
label needs to be rewritten to its per-env destination path.

_rename_builder_labels now walks two kinds of label-bearing columns:

* Built-in label arrays (body, joint, shape, articulation,
  constraint_mimic, equality_constraint), each paired with its
  matching *_world int column.
* String-typed custom-attribute columns (e.g. mujoco:tendon_label)
  paired with a references="world" companion at the same frequency.
  Any future solver-registered string column at a frequency that has
  a references="world" companion is handled automatically.

The prefix match uses a path-separator boundary
(src_path.rstrip("/") + "/") so a source path that is a string
prefix of another (/Sources/protoA vs /Sources/protoAB) does not
cross-contaminate when both feed the same envs.

The function raises ValueError if the parallel arrays of labels and
worlds have mismatched lengths, instead of silently truncating.

Adds test_rename_builder_labels.py with 10 cases covering both
passes, multi-source prefix-overlap regression, sparse env ids,
multi-frequency, multiple string columns at one frequency, and
empty-values pass-through.

Stacks on top of isaac-sim#5523 (Newton v1.2.0rc2 bump). The Newton 1.2
release also includes upstream newton-physics/newton#2659 which
scopes parse_usd's custom-frequency walk natively, removing the
cross-source MjcTendon contamination IsaacLab previously had to
work around at the framework layer.
@hujc7 hujc7 force-pushed the jichuanh/newton-1.2.0rc2-bump branch from 1c55423 to f1eadc8 Compare May 6, 2026 21:45
The Newton 1.2.0rc2 release pulls in two IsaacLab-relevant fixes
(newton-physics/newton#2678, newton-physics/newton#2720 for
SolverKamino reset under world_mask), the upstream tendon-scoping
fix from isaac-sim#2659 ("Scope USD custom-frequency parsing"), and a
VRAM-leak fix on example reset (isaac-sim#2710).

Pin bumps:

* source/isaaclab/setup.py: warp-lang==1.12.0 → >=1.13.0,
  mujoco==3.6.0 → ==3.8.0, mujoco-warp==3.6.0 → ==3.8.0.1.
* source/isaaclab_newton/setup.py: same mujoco / mujoco-warp
  bumps; Newton pin → v1.2.0rc2.
* source/isaaclab_visualizers/setup.py: 3 × Newton pin →
  v1.2.0rc2.
* tools/wheel_builder/res/python_packages.toml: same set of
  pins, mirrored across the wheel-bundling manifest.

Code adapts:

* source/isaaclab_newton/isaaclab_newton/physics/newton_manager.py
  and source/isaaclab_ov/isaaclab_ov/renderers/ovrtx_renderer_kernels.py:
  wp.math.transform_to_matrix → wp.transform_to_matrix. The
  wp.math namespace was removed in warp-lang 1.13.

Smoke verified: Shadow-Hand-Over MAPPO (4 envs, 1 iter) runs
clean on the bumped env — simulation init through CUDA graph
capture through one training step + checkpoint save, no errors.
@hujc7 hujc7 force-pushed the jichuanh/newton-1.2.0rc2-bump branch from f1eadc8 to c4c2f3d Compare May 6, 2026 21:45
Tighten the warp-lang requirement from >=1.13.0 to ==1.13.0 across
isaaclab/setup.py, tools/wheel_builder/res/python_packages.toml, and the
three changelog fragments. The Newton 1.2.0rc2 toolchain has been
validated against this exact warp-lang version; later warp-lang releases
have not been smoke-tested here.
@hujc7 hujc7 marked this pull request as ready for review May 6, 2026 22:06
@hujc7 hujc7 requested a review from hhansen-bdai as a code owner May 6, 2026 22:06
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 6, 2026

Greptile Summary

This PR bumps the Newton physics dependency from a commit-SHA pin (a27277e) to v1.2.0rc2, and carries the required transitive version updates for warp-lang (1.12→1.13), mujoco (3.6→3.8), and mujoco-warp (3.6→3.8.0.1) across all four relevant manifests. Two @wp.kernel call sites that used the removed wp.math namespace are correctly migrated to the flat wp.transform_to_matrix API.

  • Dependency pins updated consistently in source/isaaclab/setup.py, source/isaaclab_newton/setup.py, source/isaaclab_visualizers/setup.py, and tools/wheel_builder/res/python_packages.toml; no version skew between files.
  • API migration (wp.math.transform_to_matrixwp.transform_to_matrix) applied to both affected kernels (newton_manager.py and ovrtx_renderer_kernels.py); a repo-wide search confirms no remaining wp.math. usages in source files.
  • The Newton pin now uses a git tag (v1.2.0rc2) rather than the previous immutable commit SHA, trading reproducibility for readability — see inline comment.

Confidence Score: 4/5

Safe to merge; the version bumps are consistent across all manifests and the two warp API call sites are correctly updated.

The changes are well-scoped: four manifest files get synchronized version bumps and two kernel files get a straightforward one-line API rename. A repo-wide search found no remaining wp.math. usages in source. The only open question is whether the upstream newton tag v1.2.0rc2 is immutable — if it were ever re-tagged, fresh installs could silently diverge from the commit tested here.

source/isaaclab_newton/setup.py and source/isaaclab_visualizers/setup.py, where the Newton pin now references a mutable git tag rather than a commit SHA.

Important Files Changed

Filename Overview
source/isaaclab/setup.py Bumps mujoco 3.6.0→3.8.0, mujoco-warp 3.6.0→3.8.0.1, warp-lang 1.12.0→1.13.0 in INSTALL_REQUIRES; straightforward version updates.
source/isaaclab_newton/setup.py Bumps Newton pin from commit SHA a27277e to git tag v1.2.0rc2, and mujoco/mujoco-warp versions; warp-lang not explicitly pinned here (relies on Newton's transitive dep).
source/isaaclab_newton/isaaclab_newton/physics/newton_manager.py Single-line warp 1.13 API migration: wp.math.transform_to_matrix → wp.transform_to_matrix inside the _set_fabric_transforms kernel; correct and complete.
source/isaaclab_ov/isaaclab_ov/renderers/ovrtx_renderer_kernels.py Single-line warp 1.13 API migration: wp.math.transform_to_matrix → wp.transform_to_matrix in sync_newton_transforms_kernel; correct and complete.
source/isaaclab_visualizers/setup.py Updates Newton pin from commit SHA to v1.2.0rc2 tag across newton, rerun, and viser extras; no mujoco/warp-lang changes needed here (not listed in this package).
tools/wheel_builder/res/python_packages.toml Mirrors all four pin updates (warp-lang, mujoco, mujoco-warp, Newton) in both the main dep list and the newton optional group.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    subgraph Before
        A1[warp-lang==1.12.0]
        A2[mujoco==3.6.0]
        A3[mujoco-warp==3.6.0]
        A4[newton @ a27277e]
        A5[wp.math.transform_to_matrix]
    end

    subgraph After
        B1[warp-lang==1.13.0]
        B2[mujoco==3.8.0]
        B3[mujoco-warp==3.8.0.1]
        B4[newton @ v1.2.0rc2]
        B5[wp.transform_to_matrix]
    end

    subgraph Files Updated
        F1[source/isaaclab/setup.py]
        F2[source/isaaclab_newton/setup.py]
        F3[source/isaaclab_visualizers/setup.py]
        F4[tools/wheel_builder/res/python_packages.toml]
        F5[newton_manager.py\novrtx_renderer_kernels.py]
    end

    A1 -->|bumped| B1
    A2 -->|bumped| B2
    A3 -->|bumped| B3
    A4 -->|re-pinned to tag| B4
    A5 -->|wp.math removed in 1.13| B5

    B1 --> F1 & F2 & F4
    B2 --> F1 & F2 & F4
    B3 --> F1 & F2 & F4
    B4 --> F2 & F3 & F4
    B5 --> F5
Loading

Reviews (1): Last reviewed commit: "Pin warp-lang to ==1.13.0" | Re-trigger Greptile

"mujoco-warp==3.8.0.1",
"PyOpenGL-accelerate==3.1.10",
"newton @ git+https://github.com/newton-physics/newton.git@a27277ed49d6f307b8a1e4c394be7e1d14965a62",
"newton @ git+https://github.com/newton-physics/newton.git@v1.2.0rc2",
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Mutable tag replaces immutable commit SHA

The previous pin used commit hash a27277ed49d6f307b8a1e4c394be7e1d14965a62, which is immutable. The new pin v1.2.0rc2 is a git tag — if the tag is ever force-pushed on the upstream repo, CI and fresh installs would silently pull different code without any change to this file. The same applies to the three entries in source/isaaclab_visualizers/setup.py and the wheel-builder TOML. If the RC tag is expected to stay frozen (i.e. no re-tagging) this is fine as-is, but pinning to the resolved commit SHA would give stronger reproducibility guarantees.

Copy link
Copy Markdown
Contributor

@kellyguo11 kellyguo11 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isaac sim MR seems to have also updated newton and mujoco usd dependencies - https://gitlab-master.nvidia.com/omniverse/isaac/omni_isaac_sim/-/commit/43148d5d419b491ef2c4350073ef8d4928e60ef5. show we follow to do the same?

@hujc7
Copy link
Copy Markdown
Collaborator Author

hujc7 commented May 6, 2026

It looks like newton-usd-schemas and mujoco-usd-converter is transitive through newton or isaac sim. IsaacLab does not pin those.

hujc7 added 2 commits May 6, 2026 23:13
The optional [newton] extra in isaaclab_physx pulls Newton in for the
visualizer-side scene representation when the user runs PhysX with
OV/Rerun/Viser visualizers. Keep its pin synchronized with the Newton
pin in isaaclab_newton and isaaclab_visualizers; the previous commit-
SHA pin had drifted off the version used elsewhere in the repo.

Tighten the changelog wording on the existing fragments to spell out
which pins live where: warp-lang/mujoco/mujoco-warp in isaaclab + the
wheel-builder TOML; the Newton pin in isaaclab_newton, isaaclab_visualizers,
isaaclab_physx, and the wheel-builder TOML.
The changelog CLI rejects slugs containing dots — version-style
'1.2.0rc2' embedded in the slug breaks parsing. Rename to 'rc2-bump'
without dots.

AGENTS.md ## Changelog says the slug defaults to the branch name with
'/' replaced by '-', which on this branch produces a dotted slug; the
no-dots constraint lives only in tools/changelog/cli.py validation, not
in AGENTS.md. Doc fix to follow.
@kellyguo11 kellyguo11 moved this to In review in Isaac Lab May 7, 2026
@hujc7 hujc7 force-pushed the jichuanh/newton-1.2.0rc2-bump branch from 6d41e08 to 58f193e Compare May 7, 2026 20:42
Newton 1.2.0rc2 made shape-BVH construction explicit in the
SensorTiledCamera path: passing a non-None state to update() no longer
auto-builds the BVH, and the underlying RenderContext.render now raises
RuntimeError("build_bvh_shape() must be called before rendering
shapes.") if it was never built. CI's three rendering test files
(test_rendering_cartpole.py, test_rendering_dexsuite_kuka_kitless.py,
test_rendering_shadow_hand_kitless.py) all hit this on the first
newton_warp render call after the bump.

Build the shape BVH once in NewtonWarpRenderer.__init__ via
newton.geometry.build_bvh_shape(model, model.state()) — guarded by
shape_count > 0 and bvh_shapes is None so it stays idempotent across
multiple sensors that share the same Newton model. Refit per frame in
render() via newton.geometry.refit_bvh_shape against the current state,
since env body poses move every step.

Recorded in the isaaclab_newton changelog fragment alongside the
existing wp.math.transform_to_matrix → wp.transform_to_matrix adapt.
@hujc7 hujc7 force-pushed the jichuanh/newton-1.2.0rc2-bump branch from 58f193e to f8ab14e Compare May 7, 2026 20:43
),
)

# Newton ``v1.2.0rc2`` made shape-BVH construction explicit; ``SensorTiledCamera.update``
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@StafaH - looks like Jichuan included the bvh update in this PR. could you take a look to see if this is correct?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not correct but works. Ideally we want to now move the bvh build and refit outside of the renderer. In 1.2 there is a deprecation path that builds a bvh and refits it to keep old behaviour, so we can just call the original code and newton should handle building if it does not exist.

Newton 1.2.0rc2 requires warp-lang>=1.13, which dropped the warp.torch
submodule in favour of top-level warp.* (e.g. wp.torch.device_from_torch
became wp.device_from_torch). cuRobo (NVlabs/curobo) — pinned in
docker/Dockerfile.curobo and used by isaaclab_mimic — still calls
wp.torch.device_from_torch at MotionGenConfig.load_from_robot_config
time and raises AttributeError on import.

Reconstruct warp.torch as a thin forwarding module at isaaclab_mimic
import time so cuRobo's call sites keep working without an upstream
patch. Python evaluates isaaclab_mimic/__init__.py before any submodule,
including isaaclab_mimic.motion_planners.curobo.curobo_planner, so the
shim is in place before cuRobo runs. Idempotent: skipped if warp ever
re-introduces wp.torch or cuRobo migrates.

Tracking upstream at https://github.com/NVlabs/curobo — once the cuRobo
pin in docker/Dockerfile.curobo bumps to a commit that uses the
top-level wp.* API directly, drop this shim. The TODO in
source/isaaclab_mimic/isaaclab_mimic/__init__.py spells out the removal
trigger.
@hujc7 hujc7 requested a review from xyao-nv as a code owner May 7, 2026 22:26
@github-actions github-actions Bot added the isaac-mimic Related to Isaac Mimic team label May 7, 2026
@hujc7 hujc7 merged commit 9e4e62c into isaac-sim:develop May 8, 2026
54 of 56 checks passed
@github-project-automation github-project-automation Bot moved this from In review to Done in Isaac Lab May 8, 2026
AntoineRichard pushed a commit that referenced this pull request May 8, 2026
#5538)

## Summary

Two unrelated CI breakages on develop, bundled here so develop turns
green in one PR.

### 1. Skip the failing viewergl test

`test_cartpole_newton_visualizer_viewergl_rgb_motion[physx,newton]`
started returning all-black frames on develop after
`nvcr.io/nvidian/isaac-sim:latest-develop` flipped to a Kit 110.1.1 +
USD 25.11 base. The failure has been deterministic across multiple PRs
(#5523, #5495, #5408, …).

Investigation so far has ruled out:
- PR #5521 (revert in
#5539 still failed)
- Newton 1.0 → 1.2.0rc2 viewer code regression (only 7-line addition;
ViewerGL alone yields 1.08M nonzero pixels)
- warp 1.12 → 1.13 RegisteredGLBuffer ABI (byte-identical)
- Module-load side effects of `isaaclab_physx.renderers`
- CUDA-GL interop (PR #5540 diagnostic confirms direct CPU FBO readback
also returns zeros, with `GL_NO_ERROR`)
- GL context-currency (PR #5541 H6 attempt: still fails)
- GL/CUDA sync (PR #5542 H4 attempt: still fails)

Diagnostic output (PR #5540 v2):
```
[VIZDIAG] fbo=c_uint(8)  pbo=None  size=600x600
[VIZDIAG] glGetError before: GL_NO_ERROR
[VIZDIAG] CPU-readback: nonzero=0/1080000  max=0  err=GL_NO_ERROR
[VIZDIAG] PBO-result: nonzero=0/1080000  max=0
```

The FBO itself is empty — Newton's pyglet/EGL renderer is not depositing
pixels under Kit 110.1.1, even though `tiled_camera_rgb_non_black` (Kit
RTX path) on the same env passes. Underlying root cause still being
chased; this PR ships the skip to unblock develop.

### 2. Fix warp intersphinx 404 in docs build

`https://nvidia.github.io/warp/objects.inv` started returning 404 —
Warp's `objects.inv` only lives at `/stable/` and `/latest/` now. With
Sphinx's `warnings_treated_as_errors`, the broken intersphinx fetch
fails the docs build on every PR. Pinning to `/stable/` (matches the
existing PyTorch `/docs/2.11/` workaround pattern in the same file).

Verified `https://nvidia.github.io/warp/stable/objects.inv` returns 200.

## Test plan

- [x] CI `isaaclab_visualizers` on this branch — was passing earlier
with the skip; will re-verify with the bundled docs fix
- [ ] CI `Build Latest Docs` on this branch — must turn green (was
failing on every recent PR before this fix)

## Re-enable plan

Once the underlying viewergl bug is identified and fixed, drop the
`@pytest.mark.skip` decorator and remove the
`jichuanh-disable-viewergl-flaky.skip` fragment.
hujc7 added a commit to hujc7/IsaacLab that referenced this pull request May 8, 2026
After Newton's add_builder copies proto rows into the merged builder
for each environment, every row's label still references the source
proto path. Newton tracks env identity in *_world int companion
columns; IsaacLab keys data flow off USD prim paths, so each row's
label needs to be rewritten to its per-env destination path.

_rename_builder_labels now walks two kinds of label-bearing columns:

* Built-in label arrays (body, joint, shape, articulation,
  constraint_mimic, equality_constraint), each paired with its
  matching *_world int column.
* String-typed custom-attribute columns (e.g. mujoco:tendon_label)
  paired with a references="world" companion at the same frequency.
  Any future solver-registered string column at a frequency that has
  a references="world" companion is handled automatically.

The prefix match uses a path-separator boundary
(src_path.rstrip("/") + "/") so a source path that is a string
prefix of another (/Sources/protoA vs /Sources/protoAB) does not
cross-contaminate when both feed the same envs.

The function raises ValueError if the parallel arrays of labels and
worlds have mismatched lengths, instead of silently truncating.

Adds test_rename_builder_labels.py with 10 cases covering both
passes, multi-source prefix-overlap regression, sparse env ids,
multi-frequency, multiple string columns at one frequency, and
empty-values pass-through.

Stacks on top of isaac-sim#5523 (Newton v1.2.0rc2 bump). The Newton 1.2
release also includes upstream newton-physics/newton#2659 which
scopes parse_usd's custom-frequency walk natively, removing the
cross-source MjcTendon contamination IsaacLab previously had to
work around at the framework layer.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

infrastructure isaac-lab Related to Isaac Lab team isaac-mimic Related to Isaac Mimic team

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants