Skip to content

v0.1.0 — independently verified MT50 task-label audit

Latest

Choose a tag to compare

@sawhney17 sawhney17 released this 10 Aug 12:16
Immutable release. Only release title and notes can be modified.

Hash-pinned, full-corpus audit of the many-to-one task-label mapping in lerobot/metaworld_mt50.

This release:

  • projects task/index scalars from all 492 pinned Parquets and all 204,806 rows;
  • isolates the only collision: task IDs 37 and 38 share task_index 40 and one language label;
  • quantifies a stable-index repair proposal for task ID 37: 50 episodes / 8,888 rows;
  • compares 12,409 relevant state rows with a pinned MetaWorld semantic-reference layout;
  • ships a fail-closed, non-mutating sample-level preview that returns index 49 but does not append the registry row or modify Hub data;
  • passes 76 tests on Python 3.11, 3.12, and 3.13 plus independent package, provenance, tamper, and reproducibility review.

The 49-task symptom was previously reported in huggingface/lerobot#1578 by @chenkang455. This release closes the evidence and repair-scope gap; it does not claim first discovery.

Historical boundary: the exact dataset generator revisions are not established. The pinned later LeRobot integration reproduces a consistent failure mechanism, and MetaWorld 3.0.0 is used only as a semantic reference.

No missing trajectories, raw payload corruption, policy-score regression, or applied upstream dataset repair is claimed.