Skip to content

v0.3.0

Choose a tag to compare

View all tags
@allenwu-blip allenwu-blip released this 02 May 00:53
· 17 commits to main since this release
v0.3.0
35c3c4c

embodied-data v0.3.0 — minor release. Closes the largest user-facing gap from v0.2.0's Known Limitations: Beta / Alpha LeRobot v3 datasets emitted by embodied-data convert now contain re-encoded observation.images.head_color video alongside proprio.

v0.2.0 → v0.3.0 differential: v0.3.0 datasets are usable for VLA fine-tuning end-to-end. info.json.video_path is no longer null; videos/observation.images.head_color/chunk-NNN/file-NNN.mp4 actually contains a frame-aligned h264 video that downstream LeRobot v3 readers can decode. CLI surface unchanged — convert auto-detects whether the source has video and routes accordingly.

Install (or upgrade):

pip install --upgrade embodied-data

Quick start (works without any AgiBot access — uses a public LeRobot v3 dataset):

huggingface-cli download lerobot/pusht --repo-type dataset --local-dir ./pusht
embodied-data preview  ./pusht       # quick stats table
embodied-data validate ./pusht       # 5 schema/data checks → Result: PASS

Try the v0.3 AgiBot Beta path with real head_color video (requires HF access — request on the dataset card first):

# 1. Request access at https://huggingface.co/datasets/agibot-world/AgiBotWorld-Beta
# 2. Log in:
huggingface-cli login

# 3. Pull one task's task_info JSON (cheap, ~1 MB):
huggingface-cli download agibot-world/AgiBotWorld-Beta \
    --repo-type dataset \
    --include "task_info_675.json" \
    --local-dir ./agibot_beta_root

# 4. Beta proprio + per-camera mp4s ship inside multi-GB tars upstream.
#    To get one episode's proprio_stats.h5 + videos/head_color.mp4 in
#    ~10 MB without downloading the full tars, use the stream-extract
#    pattern in docs/schema/beta.md §1 (or run the reproducible script
#    scripts/fetch_beta_video_fixture.py for episode 882736 directly).
#    The episode dir layout consumed by `convert` is:
#        ./agibot_beta_root/675/882736/proprio_stats.h5
#        ./agibot_beta_root/675/882736/videos/head_color.mp4

# 5. Convert one episode (now emits real video, not video_path: null):
embodied-data convert \
    ./agibot_beta_root/675/882736 \
    /tmp/beta_v3 \
    --from agibot --to lerobot-v3

embodied-data validate /tmp/beta_v3   # frame-video alignment now actually checks → Result: PASS

# Or batch a whole task (auto-detected when src is a task root). Episodes
# without upstream head_color.mp4 are logged to .beta_batch_errors.jsonl
# and skipped — the surviving dataset still emits a clean v3 with video.
embodied-data convert ./agibot_beta_root /tmp/beta_v3_batch \
    --from agibot --to lerobot-v3

AgiBotWorld-Alpha works via the same commands — the dispatcher auto-routes Alpha-named paths through the Beta converter (schemas verified equivalent on 2026-04-30).

validate got stricter for v0.3. When info.features declares any dtype: video, frame-video alignment now hard-fails on missing mp4 files, missing episode-meta video columns, codec decode errors, or video-frame-count vs episode-length divergence >1 frame (was SKIP / silent PASS pre-v0.3). Proprio-only datasets still SKIP cleanly — no false-positive FAILs.

Repo: https://github.com/allenwu-blip/embodied-data
Schema docs: overview · DigitalWorld (sim) · Beta + Alpha · LeRobot v3 target

[0.3.0] — 2026-05-01

Minor release. Closes the largest user-facing gap from v0.2.0's Known
Limitations: Beta / Alpha LeRobot v3 datasets emitted by embodied-data convert now contain re-encoded observation.images.head_color video
alongside proprio. v0.3.0 datasets are usable for VLA fine-tuning
end-to-end — videos are no longer null.

Added

  • observation.images.head_color for Beta / Alpha. convert_agibot_beta_to_lerobot_v3
    (single-episode) and convert_agibot_beta_batch (multi-episode) now
    re-encode <src>/videos/head_color.mp4 through the LeRobot v3 video
    contract — h264, bf=0, g=2, yuv420p, monotonic PTS. Output lives
    at videos/observation.images.head_color/chunk-{i:03d}/file-{j:03d}.mp4,
    info.json declares the video feature, and per-episode video columns
    (videos/<key>/chunk_index, file_index, from_timestamp,
    to_timestamp) are emitted alongside proprio. End-to-end:
    embodied-data convert <beta_episode_dir> <dst> --from agibot --to lerobot-v3
    now produces a v3 dataset directly consumable by VLA fine-tuning
    pipelines.
  • Shared _video.py helpers. reencode_video and probe_video factor
    out the v0.1 sim re-encode path so both pipelines share a single
    enforcement of the LeRobot v3 video constraints.
  • Hard-fail on validate for declared-but-missing video. When
    info.features declares dtype: video, frame-video alignment
    now FAILs (was SKIP) on missing mp4 files, missing episode-meta video
    columns, codec decode errors, or frame-count divergence >1 frame.
    Proprio-only datasets (no dtype: video in features) still SKIP
    cleanly.

Changed

  • Beta batch all-or-nothing video. If any pending or already-committed
    episode in a batch has videos/head_color.mp4, the dataset declares
    the head_color feature; episodes lacking the upstream mp4 are logged
    to .beta_batch_errors.jsonl and skipped (preserving v0.2's
    per-episode error model). If no episode has video, output is
    proprio-only (legacy v0.2 behavior unchanged).
  • _commit_beta_episode re-encodes video before writing data. A
    missing/broken upstream mp4 fails the commit before any state-laden
    parquets land, keeping data/, meta/episodes/, and uuid_map.parquet
    in sync even on partial-failure batches.

Known limitations (still v0.3.x)

  • Multi-camera support (fisheye, hand_left/right, back_left/right).
    v0.3.0 ships head_color only — other cameras land in v0.3.1.
  • Sparse action/*/index companions still dropped silently — v0.3.2
    candidate.
  • state/end/* end-pose flattening (32-dim) — v0.3.3 candidate.
  • Reverse lerobot-v3 → agibot-beta still not implemented.
Assets 2
Loading