Smallest ManiSkill task for a cross-simulator contract pilot?

[omnilink-tech]

Hi ManiSkill team,

I maintain OmniSim, an Apache-2.0 robotics simulator with an agent-facing HTTP/MCP surface, URDF import, synthetic RGB/depth/instance output, and CPU/GPU Newton physics.

We are not asking ManiSkill to add another simulator backend. We would like to do a small portability experiment on our side using one canonical ManiSkill task:

• preserve the task's observation, action, reset, and success contracts;
• port only the minimum robot/assets with explicit provenance;
• run reproducible trials and publish the interface gaps and physical failure cases;
• return here with results before proposing any integration.

The goal is task-contract portability, not a performance comparison or a claim that OmniSim replaces SAPIEN. Which single task would be the most useful and least ambiguous target for this experiment?

Best,
Ahmed Fetouh — OmniLink

Disclosure: this note was drafted with the same AI-agent workflow used to build OmniSim and reviewed/sent by me.

[StoneT2000]

Go ahead, this is open source anyway. You can try pick cube first then peg insertion side for a more complex env.

[omnilink-tech]

Thanks — we started with PickCube as you suggested.

Reporting back before going further, because reading the contract closely turned up
three things in ManiSkill itself that are probably worth a look, and because the
blocking problems we hit are in our simulator, not yours.

Three discrepancies in PickCube / PegInsertionSide

All against main (62ff3a5), and all cheap to confirm.

1. The PickCubeEnv class attributes are dead, and one of them is off by 2x.
pick_cube.py:45-47 declares goal_thresh, cube_spawn_half_size and
cube_spawn_center, but __init__ overwrites all three from PICK_CUBE_CONFIGS
before they are ever read. The class body says cube_spawn_half_size = 0.05; the
Panda config says 0.1. Anyone reading the class to understand the task — which
is the natural thing to do — gets a spawn region half the real size.

2. The goal-height docstring excludes the cube's rest height.
The task docstring says the goal z is randomized in [0, 0.3]. pick_cube.py:129
is goal_xyz[:, 2] = torch.rand((b)) * self.max_goal_height + xyz[:, 2], and
xyz[:, 2] is cube_half_size = 0.02, so the real range is [0.02, 0.32].
Minor, but it is the kind of thing a port silently gets wrong.

3. The PegInsertionSide docstring contradicts itself about which end inserts.
Line 54 says "insert the orange end"; line 63 says "The white end of the peg is
within 0.015m of the center of the box". The code agrees with line 54:
peg_head_offsets[:, 0] = peg_half_sizes[:, 0] is +x, and +x is the half your
own comment labels # peg head and paints #EC7357. Line 63 looks like the stale one.

One more, not a bug, just easy to miss and worth stating in the docs: is_grasped
is computed and returned in info, but it is not a conjunct of success —
that is is_obj_placed & is_robot_static. We had assumed otherwise going in. A
cube that is knocked to the goal and comes to rest scores success. If that is
intentional it is fine; we mention it because every summary of this task we found,
including our own reading, assumed the grasp was required.

Where our port is currently blocked, and it is our problem

We can reproduce the scene, the randomization, the action mapping and the success
predicate. Four things in your contract have no expression in OmniSim today:

PickCube requires	OmniSim today
mu=2.0 on the two finger pads, mu=0.3 everywhere else (panda.py:20-32)	one global WorldInfo.newtonGroundMu; Solid has no friction field
link.disable_gravity = True on every Panda link (base_agent.py:278-283)	no per-body gravity control at all
patch_radius=0.1 on the pads (torsional friction)	newtonCondim, documented in our own schema as "GLOBAL, not per-shape"
<mimic> on the fingers, plus a PhysX fixed tendon at 1e5	no mimic or tendon support in our URDF import

The friction one is the real wall. A 6.7x ratio between the pads and the table is
not a detail — it is what lets a 0.064 kg cube be lifted without slipping while
still sliding on the table. With a single global mu we get one of those two wrong
by construction, so any success rate we published today would be measuring our
missing feature rather than the task. We would rather say that than report a number.

The gravity one is nearly as bad: with the arm gravity-free, your PD gains
(kp=1e3, kd=1e2, force_limit=100) are tuned for a world we cannot currently
reproduce, so the arm sags under the same gains and every trajectory diverges.

Two things we expected to be problems and are not, which may be useful to you:
the PegInsertionSide hole is four convex boxes on one kinematic body, and the table
is a single box collider with add_nonconvex_collision_from_file commented out at
scene_builder.py:27-31. We have measured that MuJoCo silently convexifies every
triangle-mesh collider — a bowl modelled as one concave mesh collides as a solid
lump — so a hole authored as a mesh would have been unportable and, more to the
point, would already behave wrong in any MuJoCo-family sim. Yours sidesteps it.

What we are doing next

Adding per-material friction is the prerequisite, and it is on us. We will come back
with actual trials once it lands, rather than post a success rate that is really a
measurement of our own gap. If per-episode results at matched initial conditions
would be more useful to you than aggregate success rates — i.e. we replay your
sampled cube/goal poses rather than trying to reproduce the torch+numpy RNG streams
— say so and we will structure it that way.

No integration proposal attached, and none intended until there are numbers.

OmniLink

[sattyamjjain]

The three mismatches you list are exactly the kinds of issues that break cross-simulator comparison, and I would argue they break same-simulator comparison too.

If the spawn region is 2x the documented size, then two runs of the same checkpoint are not sampling the same task, and any difference between them absorbs that variance silently. It shows up as noise in a success rate, and people widen their error bars instead of fixing the contract.

What helped me was treating the task contract as an artifact rather than documentation. Spawn bounds, goal tolerance, success predicate, and the seed all get written into the run output. Then a disagreement between two results is diagnosable instead of being a vibe.

For the pilot, the smallest useful task is probably the one where the success predicate is a single geometric condition with no tolerance ambiguity. The peg case is a bad first pick precisely because of the docstring contradiction you found.