Skip to content

Releases: Alecbossard/GeoPolicy-Bench

V4 — RGB-D robustness and reproducible checkpoint demo

Choose a tag to compare

@Alecbossard Alecbossard released this 06 Oct 08:41

GeoPolicy-Bench V4 — RGB-D robustness

Compact checkpoint and source bundle for a reproducible MuJoCo Panda placement demo.

  • V3: 147/150 stable placements (98%) on a one-cube, one-tray, fixed-instruction task, across three training seeds.
  • V4: fusion without augmentation achieves 483/540 (89.4%), versus fixed-only 272/540 (50.4%), over nine synthetic perturbation settings. Paired descriptive 95% interval for the difference: +39.07 pp [+35.00, +42.78].
  • Negative result: the tested camera/depth augmentations show no demonstrated overall gain.

V3 and V4 use different reserved tests and are not a direct before/after comparison. V4 includes 12 matched behavior-cloning policies and 2,400 reserved-test rollouts; training used 79 successful demonstrations from 80 collections. No physical robot or real sim-to-real transfer was evaluated.

Download and replay

Download geopolicy-v4-demo.zip (1,602,161 bytes) and verify SHA-256:

c609fce309c21b9293cd868e90243d4c0815cf53e9a27b9f79479ce7422d9b05

Extract into a fresh directory. Validated setup: Windows, Python 3.11.9 and RTX 4060 Laptop 8 GB. The ZIP contains source, compact checkpoint with normalization, exact pins and expected observations/actions/physics. It needs no training HDF5 data. Inference uses CPU; NVIDIA rendering/resource checks are still required.

python -m venv .venv
.venv\Scripts\python.exe -m pip install -r requirements-lock.txt --extra-index-url https://download.pytorch.org/whl/cu128
.venv\Scripts\python.exe scripts/v4/demo.py replay

The 5.5-second demo is fusion with augmentation, seed 0, fixed camera absent, the first reserved scene 500000. This case was predeclared. It illustrates replay, rather than the unaugmented model's aggregate result. Exact replay was verified in a separate source copy and second pinned environment on the same PC; bitwise portability to other systems is untested.

Nine release files include the ZIP, separate checkpoint, MP4/GIF, dependency pins, runtime/checkpoint/package manifests and SHA256SUMS.txt. The package manifest describes the release files; the repository manifest records the current publication and verification status. SHA256SUMS.txt covers the other eight assets.

README · Release/reproduction guide · V3 report · V4 report

Manual color prior, known objects, ideal simulated calibration and perturbations applied to sampled points limit the findings. Initial QC matches both camera representations before view selection; it does not claim identical selected inputs or subsequent trajectories. All negative results and raw sensor exceptions remain documented. Historical V1/V2/V3 artifacts are preserved.

V2 demo — camera fusion study and reproducible Panda placement

Choose a tag to compare

@Alecbossard Alecbossard released this 01 Oct 18:00

This release provides the compact EMA checkpoint and raw expected examples for the verified V2 Panda manipulation study. A fresh clone plus this 4.3 MB bundle can replay the disclosed success and failure without training data.

The repository compares fixed, wrist and fused RGB-D views and the manual color prior using 21 models, three training seeds and 3,810 final rollouts. Results remain limited: nominal V2 fusion scored 10/150 versus preserved V1 16/150 on the same stable criterion, and selected ACT scored 0/150. No nominal performance gain is claimed.

Included asset: demo_bundle.zip — selected fusion-prior seed-0 EMA policy, recipe, identities, expected outcomes and per-step demo traces. Its SHA-256 and member checksums are in docs/releases/v2_demo.json. The archive is byte-identical to the verified local delivery. Full datasets, training/resume checkpoints and VLA assets are not included.

Validated locally on Windows, Python 3.11 and RTX 4060 Laptop 8 GB. Separate pinned and source-and-bundle-only replays reproduced every action and selected-object position exactly. Follow the main README or docs/v2_reproduction.md; use python scripts/download_demo.py to verify and extract the release asset.

Scientific snapshots remain under geopolicy-v1-2026-09-30 and geopolicy-v2-2026-10-01. This v2-demo tag adds the curated GitHub presentation and release download helper, without changing frozen models, normalization, evaluator code or recorded outcomes.