VSArena 1.0.0

VSArena 1.0.0

The official open benchmark for embodied (VLA) policies: one stacking task, one protocol, one public board the browser cannot write.

What’s in

• Studio — browser work cell (Rapier 60 Hz) for teleop, demos, and debug
• Official eval — hosted harness writes ELO; Studio can wake & spectate (read-only)
• Integrity — weekly sampler seed, control arm, SHA-256 digest + Ed25519 receipts
• VLA track — RGB 128×128 + instruction; cube poses hidden from the policy
• Python SDK — dry-run locally, live against the harness (sdk/python)
• Account — GitHub sign-in after accepting Terms & Privacy; API key + agent name on /account
• Leaderboard — public ranking for official VLA matches only

Links

• Site: https://vsarena.vercel.app
• Leaderboard: https://vsarena.vercel.app/leaderboard
• Submit guide: https://vsarena.vercel.app/submit
• Protocol: docs/harness.md
• Eval integrity: docs/eval-integrity.md
• SDK: docs/sdk.md

Not in this release

• Multi-task suite
• Head-to-head 1v1 Arena product
• PyPI package (SDK ships in-repo)

Note

Studio demos, teleop, Baseline-IK / ColorSeek in the tab, and dry_run=True do not update public ELO. Only live VLA matches through the official harness do.

Studio v0.6.0

Studio v0.6.0 — 6 Sep 2026

Official matches now carry a fixed seed, a control run, and a signed receipt. Ingest rejects unsigned or tampered results. Deploy the Next app and the harness together.

The arm is redesigned (look only — 4-DOF and grasp physics are unchanged). The leaderboard is cleaner. Account is a desk: key, agent name, public card, and a Live / Practice rail.

Studio demos and dry_run=True still do not write public ELO.

Protocol

Unchanged for clients: hello → state → action → result. joint_1…joint_4, ee_delta, VLA RGB + instruction.

Upgrade

1. Deploy Next and npm run harness / the hosted harness on the same release.
2. Keep HARNESS_INGEST_SECRET (≥16) on both.
3. Optional: VSARENA_RESULTS_SIGNING_KEY if you want a dedicated HMAC key.

Not in this release

Arena 1v1 and Community are still soon.

Eval integrity: taxonomy, held-out scenes, provenance

Official matches now say why they ended, which simulator ran, and which layout was used — without pretending the in-repo held-out set is secret.

Have

• Failure taxonomy on error / result: policy.* vs protocol.* vs harness.*
• Provenance stamp: product 0.5.0, Rapier 0.20.0, 60 Hz, git SHA, scene id/seed/hash
• Action contract: invalid joints / huge ee_delta never touch Rapier (5 strikes → abort)
• Latency budgets: VLA 2 s (8 strikes), state 150 ms (20 strikes) → policy.timeout
• Sparse replay artifact vsarena-replay-v1 on the result (not stored in Postgres)
• Disconnect mid-match does not write ELO

Scenes

• Studio stays public canonical
• Production harness defaults to held_out (8 in-repo layouts + jitter)
• Operators can inject a private set with VSARENA_HELD_OUT_JSON

Honest limit: those eight layouts live in git. They are a different public set, not a secret exam. True privacy needs the JSON override on the host.

Docs

• docs/eval-integrity.md
• Protocol extras on result.failure / result.provenance / result.replay

Studio branding is v0.5.0. Python SDK 0.5.0 (parse_action_contract, format_harness_error).