Full-corpus VLABench contract audit and explicit-open repair
This release pins and scans all 3,114,872 action rows in
lerobot/vlabench_unified@f61ad898199e746c20af3112f278cc86eb3ca3c4 against
LeRobot's default eef environment contract at
22bd7a2f489b367d8df42de803b1e8c4ca63a3f9.
Exact findings
- 3,114,872 / 3,114,872 rows fall outside the declared
[-1,-1,-1,-1,-1,-1,0]to[1,1,1,1,1,1,1]Box. - 5,545,679 / 21,804,104 scalar action coordinates fall outside it.
- All 10,977 episodes and 295 task strings contain at least one rejected action.
- A broad representation-level control for absolute XYZ, Euler radians, and binary gripper
contains every action row; it is not claimed as a calibrated physical workspace. - The stored state gripper bit follows a legacy closed predicate, while the action bit uses
1 = open. Same-row complement agreement is 95.2043%; previous-action complement agreement
is 96.2899% on comparable rows.
The trajectories are not shown to be corrupt. The pinned wrapper consumes these values as
absolute robot-frame pose targets through inverse kinematics. No rollout failure, policy-score
change, image defect, or safe silent checkpoint migration is claimed.
Repair asset
The explicit-open-v2 sidecar provides 3,114,872 keyed rows without modifying source Parquets or
actions. Consumers must explicitly choose legacy_closed_v1 or explicit_open_v2; automatic
polarity guessing is forbidden. The release includes all three sidecar shards, their manifest,
the exact audit evidence, source manifest, wheel, and source distribution.
Reproducibility
- Reviewed commit:
d5de59bcfd8be90b31cc58af8480bf92c8ed50b7. - 81 tests pass on Python 3.10, 3.12, and 3.13.
- The full scan reproduces byte-identically at
3e33d42186ca0b4058aca1551753d72f04932e362455096112adb25223316272. - The complete sidecar manifest is
bdddd0f505242151f158a9652ae27a9248088acedd87d951e0c49e527b4329c1. - Wheel and source distributions build deterministically from the reviewed commit.
Public demo: https://sawhney17.github.io/vlabench-contract-audit/demo/
Upstream LeRobot action-contract report:
huggingface/lerobot#4406
Upstream VLABench gripper-convention report:
OpenMOSS/VLABench#88
Both reports require maintainer review. No merge or upstream acceptance is claimed.