v0.5.0

RoboDojo, the RoboCasa/RoboCasa365 split, and a CPU render backend lead v0.5.0.

RoboDojo (#101)

• New robodojo benchmark adapter and pi0.5 RoboDojo model server in configs/model_servers/robodojo_pi05.
• Isaac Sim 5.1 based. The 36.3 GB image is build-locally-only behind the NVIDIA EULA gate: docker/build.sh robodojo --accept-license robodojo.
• One process evaluates exactly one task because Isaac Lab's SimulationContext is process-global and RoboDojo environment teardown hangs.
• scripts/run_robodojo_protocol.sh reads the protocol task table, materializes a single-task config for each task, and runs them sequentially. Run several instances with disjoint --filter values for parallelism, one lane per GPU.

RoboCasa and RoboCasa365 (#103)

• RoboCasa v0.2 and RoboCasa365 v1.0.1 are now separate benchmarks with separate configs and images.
• robocasa implements the original RoboCasa release: 24 atomic tasks, 7-D delta pose actions, the v0.2 pin, obj_instance_split, held-out object split B with five fixed layout/style pairs, and registry horizons of 300 to 1000 steps.
• robocasa365 implements the incompatible v1.0.1 release: target50 tasks and 12-D Panda-Omron actions.
• The robocasa image is now 21.4 GB instead of 42.4 GB; the new robocasa365 image is 35.6 GB.

CPU rendering and RoboMME policy (#106, #115)

• New render: cpu config key and --render cpu CLI flag start benchmark containers without a GPU and use software rendering.
• Combining --render cpu with explicit --gpus or a config demanding GPU rendering is rejected. CPU-rendered shard workers are not sized by GPU count.
• scripts/run_sharded.sh --render gpu|cpu forwards the selected backend to every shard.
• RoboMME now defaults explicitly to render: cpu. The previous ROBOMME_USE_LAVAPIPE=auto probe could miss failures under combined CUDA and SAPIEN rendering (#112).
• Known-good hosts can opt into the approximately 5-10x faster RoboMME simulator path with --render gpu.
• BEHAVIOR-1K is labeled as implementing the 2025 challenge protocol (#113); the 2026 evaluator path is tracked in #114.

Public image pull fallback (#118)

• A failed pull from ghcr.io/allenai/vla-evaluation-harness/* is retried against ghcr.io/worv-ai/vla-evaluation-harness-public/*.
• The mirrored image is retagged under its canonical name, so configs and the local-image cache continue to work unchanged.
• This addresses allenai org package grants lagging behind releases (#8, #73).

Model server and sharding fixes (#124, #125)

• The model server now imports websockets.asyncio.server.serve directly, avoiding HTTP 500 failures on WebSocket handshakes, /health, and /config when the legacy server alias was selected.
• The websockets dependency floor moves from >=12.0 to >=13.0, the first version containing that module. Python 3.8 can still resolve websockets 13.1. Reported with full diagnosis by @rakhimovv.
• Shards now run in separate process groups, and interruption cleanup signals those groups instead of relying on the parent shell being a process-group leader. This prevents orphaned shards from retaining GPUs and benchmark containers under CI, containers, schedulers, and run_simpler_seeds.sh (#122). Diagnosed by @rakhimovv, credited as co-author.

Docker image repairs (#90, #91, #93, #107)

• RLBench pins the moved stepjam repositories by commit SHA and points QT_PLUGIN_PATH at CoppeliaSim's bundled Qt GLX plugin, fixing black-frame renders. The image remains build-locally-only.
• LIBERO, LIBERO-Mem, LIBERO-Plus, LIBERO-Pro, and RoboCerebra use editable_mode=compat, fixing empty import libero results from the PEP 660 editable finder (#92).
• simpler pins scikit-build-core so the ruckig build continues to work.
• behavior1k resolves omniverse-kit across package indexes.

Smaller fixes (#97, #98, #100)

• The recording video writer is importable on Python 3.8 and 3.9 again (#95).
• typing_extensions is declared for Python versions below 3.11 to provide the NotRequired backport.
• vla-eval run pins output_dir to an absolute path, and smoke tests fail when the result JSON is missing instead of passing vacuously.

⚠️ Breaking changes

• RoboCasa is split into robocasa and robocasa365 (#103). Users with old robocasa configs must choose which benchmark those configs were intended to evaluate.
• The websockets dependency floor is now >=13.0 (#124).
• RoboMME now defaults to CPU rendering (#115). Use --render gpu on known-good hosts to restore GPU rendering.

Maintenance

• GitHub Actions updates: setup-python 7 and labeler 7 (#104, #105).
• Leaderboard content and pipeline updates continue to ship through Pages and are not part of the package release (#94, #102, #108, #109, #117, #119).

Full changelog: v0.4.0...v0.5.0