CLEAR-LeWM v0.3.0: task-semantic evaluation

CLEAR-LeWM v0.3.0 makes benchmark success correspond to physical task completion across all four LeWM tasks.

Highlights:

• Two auditable protocols: Moderate for continuity and Strict for stronger task completion claims
• PushT object-only pose evaluation that excludes the pusher endpoint
• Cube position plus symmetry-aware orientation over all 24 proper cube rotations
• Reacher wrapped joint geometry with holding and terminal speed reported separately
• TwoRoom swept-circle collision, full-radius door clearance, and mandatory legal-route validation
• Deterministic versioned manifests with matched random-policy controls
• Historical Official, Moderate, and Strict outputs with checkpoint and runtime provenance
• Official high-epoch LeWM checkpoint registry and strict checkpoint preflight
• Rebuilt 1080p rule-comparison film, high-resolution task GIFs, and task-specific evaluation guides

Reference results (LeWM / random, 100 pairs, seed 42, 300 x 30 CEM, solver batch 1):

• PushT Strict: 79% / 2%
• Cube Strict: 18% / 3%
• Reacher Strict: 36% / 4%
• TwoRoom Strict: 24% / 0%, with 0/100 invalid routes

Compatibility:

• v0.1 and v0.2 artifacts remain executable and are not silently rewritten.
• v0.3 protocols, manifests, checkpoint registry, and results are versioned explicitly.

Validation:

• 37 tests passed; 2 optional-dependency tests skipped
• Ruff lint and repository-wide formatting checks passed
• README local links validated
• All release MP4/GIF media fully decoded successfully

CLEAR-LeWM v0.2.1: audited FAST and runtime modes

This patch release makes performance behavior explicit, measurable, and auditable.

Highlights:

• README FAST section with exact correctness fixes and measured speedups
• 9.25x loader-only PushT throughput on the H20-3 node (4661 vs 504 samples/s)
• Historical 1.8x end-to-end H200 training observation (11.0 vs 6.1 steps/s)
• Safe FAST resume validation, batched source fetches, and vectorized memmap writes
• Reproducible FAST audit and isolated benchmark script
• Opt-in CEM environment batching with a canonical LeWM broadcast fix
• H20-3e CEM sweep: batch 16 reduced solver time from 112.8s to 75.6s (1.49x)
• Explicit warning that batched CEM changes random-sample assignment and must not replace batch-1 reference results
• CPU thread and matmul precision controls with all runtime settings/timings recorded in result JSON
• Structured benchmark records and a detailed PERFORMANCE.md audit

Validation:

• 17 base tests passed; one PyTorch-only test skipped in the lightweight environment
• 5/5 FAST and batched-criterion tests passed in the full LeWM/PyTorch environment
• Ruff, formatting, privacy scan, JSON validation, and wheel build passed

The official v0.2.0 reference tables remain unchanged.

CLEAR-LeWM v0.2.0: calibrated three-tier evaluation

CLEAR-LeWM v0.2.0 establishes a reproducible three-tier LeWM evaluation standard.

Highlights:

• Official, Moderate, and Strict protocols for PushT, Reacher, TwoRoom, and Cube
• Fixed 100-pair seed-42 manifests and 24 checked-in random/official-LeWM reference runs
• All four revision-pinned official checkpoints strictly load 303/303 tensors and remain above random in every tier
• Corrected task success predicates, episode-balanced sampling, initial-success filtering, and strict difficulty floors
• Auditable official-checkpoint preparation with source and runtime SHA-256 provenance
• Corrected FAST memmap loader plus conversion, resume validation, and source-equivalence audit
• Canonical HDF5/Lance data-format contract and a refreshed protocol GIF/README

Reference SR (Official / Moderate / Strict, official LeWM):

• PushT: 89% / 74% / 42%
• Reacher: 87% / 63% / 22%
• TwoRoom: 85% / 70% / 41%
• Cube: 62% / 36% / 17%

Validation: 17 tests passed, Ruff clean, wheel build verified.

CLEAR-LeWM v0.1.0

Initial public release of the CLEAR-LeWM evaluation standard.\n\nHighlights:\n- official-compatible, strict in-distribution, held-out, and hard evaluation tracks\n- deterministic episode-balanced manifests for four LeWM tasks\n- initial-success filtering and paired random-adjusted metrics\n- strict Cube pose success with a five-step hold requirement\n- pinned public LeWM upstream source\n- 16 reference manifests and four deterministic random baselines\n\nValidated with 11 synthetic regression tests and end-to-end OGBench-Cube evaluation.