Skip to content

feat(annotate_subtasks): 448x448 frames; select camera0 from standard mapping#257

Merged
shuheng-liu merged 1 commit into
mainfrom
feat/annotate-subtasks-camera0-448
May 4, 2026
Merged

feat(annotate_subtasks): 448x448 frames; select camera0 from standard mapping#257
shuheng-liu merged 1 commit into
mainfrom
feat/annotate-subtasks-camera0-448

Conversation

@WilliamYue37
Copy link
Copy Markdown
Member

What this does

Two changes to src/opentau/scripts/annotate_subtasks.py (🗃️ Feature):

  1. Default frame resolution → 448 × 448 square. Replaces --target-width 640 with --target-size 448. Frames are now scaled so the shorter side equals target_size, then center-cropped to target_size × target_size, matching standard VLM preprocessing. Already-square inputs at the target resolution pass through untouched.

  2. camera0 selection from the standard data format mapping. The dataset's camera0 video feature is resolved with this priority:

    1. DatasetConfig.data_features_name_mapping["camera0"] inline on the mixture config (src/opentau/configs/default.py:101),
    2. DATA_FEATURES_NAME_MAPPING[repo_id]["camera0"] from src/opentau/datasets/standard_data_format_mapping.py,
    3. Fallback: first feature in info.json with dtype=='video' (the script's pre-existing behavior).

    Datasets whose camera0 resolves to a feature that is missing from info.json or has dtype != 'video' (e.g. the dummy / vsr / cocoqa rows that map camera0 → "image") are skipped with a warning — they are not LeRobot video datasets.

The module docstring and CLI --help text are updated to match.

How it was tested

  • Added TestResizeAndCenterCrop (5 cases: landscape / portrait / already-square / upscale / non-default size) and TestResolveCamera0VideoKey (8 cases: inline-vs-global precedence, fallback paths, non-video feature, missing feature, missing repo_id, no-video-features) in tests/scripts/test_annotate_subtasks.py.
  • All 46 tests pass:
    pytest tests/scripts/test_annotate_subtasks.py
    
  • Pre-commit (ruff lint+format, pyupgrade, bandit, license header, etc.) all green on both changed files.

How to checkout & try? (for the reviewer)

pytest -sx tests/scripts/test_annotate_subtasks.py

# Dry run on one episode of a dataset whose mixture config has either an
# inline `data_features_name_mapping` or a repo_id present in
# DATA_FEATURES_NAME_MAPPING:
python src/opentau/scripts/annotate_subtasks.py \
    --config-path configs/examples/train_mixture_config.json \
    --max-episodes-per-dataset 1 \
    --no-write-response-column

Checklist

  • I have added Google-style docstrings to important functions and ensured function parameters are typed.
  • My PR includes policy-related changes.
    • If the above is checked: I have run the GPU pytests (pytest -m "gpu") and regression tests.

🤖 Generated with Claude Code

…tandard mapping

- Replace --target-width (640) with --target-size (448) and resize each frame
  by scaling the shorter side to target_size + center-cropping to a square,
  matching the standard VLM preprocessing pipeline.
- Resolve the per-dataset video feature from camera0 of the standard data
  format mapping (DatasetConfig.data_features_name_mapping inline override
  in the mixture config first, then DATA_FEATURES_NAME_MAPPING globally).
  Fall back to the first dtype=='video' feature in info.json when no
  mapping exists; skip datasets whose camera0 resolves to a non-video
  feature (not a LeRobot video dataset).
- Add tests for _resize_and_center_crop and _resolve_camera0_video_key.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown
Contributor

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed — no inline findings. Summary in PR comment.

@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 4, 2026

[claude-review] summary for commit 6e0380c

No blocking issues found.

@shuheng-liu shuheng-liu self-requested a review May 4, 2026 21:55
@shuheng-liu shuheng-liu merged commit e96e9e6 into main May 4, 2026
8 checks passed
@shuheng-liu shuheng-liu deleted the feat/annotate-subtasks-camera0-448 branch May 4, 2026 21:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants