cookbook(action): repair the DROID LeRobot example asset

[lfengad]

Follow-up to the DROID breakage reported on #299. The bundled droid_lerobot_example asset is broken in five independent ways — the two errors in that thread are just the first two you hit.

Root cause

The asset was hand-assembled with pandas/pyarrow from a LeRobot v2-era template, instead of being written through LeRobot's own dataset writer. Everything below follows from that.

#	Defect	Error
1	info.json declares 4 features as shape: []	ValueError: Corresponding feature is not valid
2	meta/episodes/*.parquet missing v3 routing columns	KeyError: 'videos/.../chunk_index'
3	info.json uses v2 path templates ({episode_chunk})	KeyError: 'episode_chunk'
4	tasks.parquet stores task text as a column, not the index	AttributeError: 'int' object has no attribute 'split'
5	Frame data coerced to match the wrong shapes in (1)	data loss

Sibling assets built by the same exporter carry the same signature (v2 templates, no routing columns, non-indexed tasks.parquet): agibotworld_beta_lerobot_example, bridge_lerobot_example, robomind_lerobot_example/franka_dual. None have shape: [] or collapsed columns and they load today, so they are latent rather than broken — flagging for the same owner, not fixed here.

Defect 5 is data loss, not a metadata mismatch

Diffed against the source dataset recorded in info.json's provenance fields (droid_plus_lerobot_640x360_20260412/success, episode 48330). The corruption follows an exact rule:

example[i] == source_frame0.ravel()[clip(i, 0, dim-1)]

i.e. frame 0's vector spread over the first dim rows, then edge-padded for the remaining 700+:

column	source	shipped
observation.state.joint_positions	(722, 7)	(722,), 7 unique values total
action.joint_position	(722, 7)	(722,), 7 unique values total
observation.state.gripper_position	429 non-zero	all zero

The coercion was not applied uniformly — action.gripper_position and observation.state.cartesian_position came through intact despite action.gripper_position also being declared shape: []. So correct declarations do not imply correct data here; all five kept columns were re-derived from the source rather than only the failing ones.

Changes

Asset regenerated from the source. MP4s untouched — info.json records them as frame-exact (first_frame_mae_uint8_vs_exact_source: 0.0) and that was verified.

Notebook DROID cell switched to DROIDMergedLeRobotDataset. DROIDLeRobotDataset derives its feature layout from os.path.basename(root), which a bundled example directory can never satisfy:

ValueError: Unknown version: 'droid_lerobot_example'

DROIDMergedLeRobotDataset already auto-discovers the layout for arbitrary local roots, so the symlink shim used in #299's diffusers notebook is not needed — that notebook can drop its shim too.

Verification

All five columns now match the source exactly:

observation.state.cartesian_position  (722, 6)  match=True  unique_rows=722
observation.state.joint_positions     (722, 7)  match=True  unique_rows=722
observation.state.gripper_position     (722,)   match=True  unique_rows=103
action.gripper_position                (722,)   match=True  unique_rows=125
action.joint_position                 (722, 7)  match=True  unique_rows=722

End-to-end on GB200 (driver 580.126.20) with Cosmos3-Edge, running the notebook's DROID section:

[1/3] cell 32 -> records: 5
[2/3] cell 34 -> 5/5 chunks generated
[3/3] cell 36 -> STITCHED (80, 528, 640, 3) fps=15
      seam 0->1 mean_abs_diff=3.19   seam 1->2 3.04
      seam 2->3 3.21                 seam 3->4 2.72
      std over time 13.89

Autoregressive seams are continuous (no jump), the video actually evolves, and the concat_view layout is intact. Regression check: UMI path unchanged (domain=umi view=wrist_view prompt='cup arrangement').

Known limitation (not addressed here)

This notebook cannot be executed headlessly. configure_cosmos_framework_runtime_env() re-execs the kernel via os.execv so torchcodec can dlopen FFmpeg, and under nbconvert/papermill nothing reconnects to the re-exec'd kernel — the client waits forever for a reply to a cell that no longer exists, with no output and no GPU activity.

The restart itself is unavoidable: the dynamic linker fixes the library search path at process start, and preloading is not an alternative because PyAV ships its FFmpeg with hashed sonames (libavutil-fc54c7f1.so.60.8.100) while torchcodec asks for the plain libavutil.so.60, so dlopen never matches an already-loaded copy.

Interactive use is unaffected — the frontend reconnects and the notebook tells you to re-run Step 2. Since this repo has no notebook CI today, nothing is changed here. To drive it headlessly, export the paths the function would have set plus COSMOS3_LD_LIBRARY_PATH_READY=1 before starting the kernel (that is how the verification below was run). The real fix, if cookbook CI is ever wanted, is to decode with lerobot's pyav backend — PyAV resolves its own libraries via RPATH=$ORIGIN/../../av.libs and needs no environment setup — which would let the whole LD_LIBRARY_PATH + restart mechanism be deleted. That needs video_backend plumbed through the public dataset classes in cosmos-framework.

Depends on

NVIDIA/cosmos-framework#156 — must land and ship first. create_record_from_dataset reads dataset.domain_name / dataset.viewpoint, which BaseActionLeRobotDataset does not expose. Merging this PR alone turns the failure from ValueError: Unknown version into AttributeError: ... has no attribute 'domain_name'.

Known issue, not fixed here

The generated record carries prompt: ''. This is not an export bug — DROID episode 48330's task text is " | | " in the source dataset itself, and HAS_MULTI_LANGUAGE_ANNOTATIONS=True splits on " | ". Fixing it means picking a different episode and re-cutting the MP4s, which is a call for whoever owns the DROID slice (see the provenance fields in info.json).

🤖 Generated with Claude Code

[MaciejBalaNV]

@lfengad
I get this error in the Create the DROID Autoregressive Forward-Dynamics Plan cell right now. I'm using the latest cosmos-framework version:

      7 droid_dataset = DROIDMergedLeRobotDataset(root=droid_dataset_root, chunk_length=chunk_length)
----> 8 droid_records = create_record_from_dataset(droid_dataset, num_chunks, chunk_length)
     10 domain_name = droid_dataset.domain_name
     11 droid_fd_input_path = COSMOS3_INPUT_DIR [/](http://127.0.0.1:8888/) f"action_forward_dynamics_{domain_name}_custom.jsonl"

Cell In[4], line 127, in create_record_from_dataset(dataset, num_chunks, chunk_length)
    125 for chunk_idx, sample_idx in enumerate(chunk_starts):
    126     data_sample = dataset[sample_idx]
--> 127     domain_name = dataset.domain_name
    128     chunk_name = f"{domain_name}_id_chunk_{chunk_idx:02d}"
    130     action_path = COSMOS3_INPUT_DIR [/](http://127.0.0.1:8888/) f"{chunk_name}.json"

AttributeError: 'DROIDMergedLeRobotDataset' object has no attribute 'domain_name'

[MaciejBalaNV]

I confirm it works fine now.