RAX: Scaling Online Exploration for Robot Agents

RAX: Scaling Online Exploration for Robot Agents
Runhan Huang^1*, Xiangcheng Zhang^1*†, Haoyu Chen^1‡, Yitong Li^2‡, Yen-Jen Wang^3‡, Bohan Lyu^3‡, Haotian Ye⁵, Chuye Hong³, Yixuan Wang², Chloe Su¹, Haoyu Xiong⁴, Yangzhen Wu³, Enze Xie², Travis Manderson⁴, Stefano Ermon⁵, Pieter Abbeel³, Antonio Torralba⁴, Dawn Song³, Weirui Ye⁴, Yilun Du^1
1Harvard University   ²NVIDIA   ³UC Berkeley   ⁴MIT   ⁵Stanford University
^*Equal contribution   ^†Project lead   ^‡Core contributor

• [2026-10-21] 🔥 DeepSeek and Kimi are here, and you can serve the model yourself! The simulation branches now run DeepSeek Harness and Kimi Code alongside Codex CLI and Claude Code, natively or in a container. Try local serving: three commands prepare, serve and validate DeepSeek-V4.1-Flash with vLLM on one node with four H200s, and Claude Code or DeepSeek Harness then drives the robot through your own endpoint, no hosted API needed. Kimi Code runs through OpenRouter, the Moonshot API, a Kimi account or any compatible endpoint. 👉 Self-host DeepSeek · Agent harnesses · DeepSeek model results
• [2026-10-17] 🌿 HumanoidArena and DexJoCo branches released, each as a container and a native branch. See Branches.
• [2026-10-16] 🗂️ All real-robot sessions released on Hugging Face. robot-session-logs holds every Franka and RB-Y1 session: the agent's full transcripts, every tool call, robot state, camera recordings and the code and skills it wrote. See Session logs.
• [2026-10-15] 📄 The paper is on arXiv: 2610.22496.
• [2026-10-13] 🌿 RoboDojo, BEHAVIOR-1K and real-robot branches released: simulation as a container and a native branch, and the real Franka FR3 and Rainbow RB-Y1 harnesses.

RAX (Robot Agent eXploration) is an open-source framework for robot agents built around scaling exploration. A general-purpose coding agent (Codex CLI or Claude Code, with a frozen model) drives a robot through perception and control tools, learns the task from its own online experience, and compiles what worked into a modular robot program that runs without the agent.

• 🚫 No demonstrations. No action demonstration data, hand-engineered action primitives or value-function guidance.
• 📈 Exploration scales. Generalization to unseen environments improves consistently as online experience grows.
• ⚡ Exploration compiles into code. The program the agent writes runs up to 241× faster than the agent itself.
• 🤖 One harness, every body. Single-arm, bimanual, dexterous, mobile and humanoid robots, in simulation and on real hardware. New robots plug in through their API alone.

• 📈 Scaling exploration. Mean success on unseen scenes rises from 25% to 79% as the exploration budget grows from 4 to 128 episodes, against 41.2 for the best prior SOTA.
• 🖥️ Simulation. RAX beats the top baseline models by +37.4% on average.
• 🦾 Real robots. Learns memory-dependent, articulated, high-precision, deformable and long-horizon mobile manipulation tasks without human demonstrations.

Exploration compiles into code. The program RAX writes replaces the agent at run time:

The simulation branches support Codex CLI, Claude Code, DeepSeek Harness and Kimi Code in both native and container execution. Choose the harness that manages the agent's reasoning and tools, then connect it to OpenRouter, a supported vendor account or API, or your own model endpoint.

1. Explore. The agent solves the task by hand from observations, calling the robot's tools one step at a time, and records what worked (preconditions, actions and effects) in its memory.
2. Compile. It turns the successful decision sequence into a program, controller.py:main(ctx), made of step functions that check what they observe, and tests it unattended.
3. Evaluate. In simulation, the frozen program, or a live copy of the agent, plays held-out layouts it has never seen. On hardware, the operator resets the scene and the program runs end to end.

Everything the agent learns lives in its workspace: TASK.md, AGENTS.md, skills/, memory/ (notes, tips, verified skills) and code/<project>/controller.py, the compiled program.

The harness (workspace generator, skills, launcher, sandbox, MCP frontend) never imports a robot or simulator package. Each simulator or robot is a backend behind the same tool interface, so one agent, one set of skills and one workflow run on every embodiment.

coding agent (Codex / Claude Code)
  in an offline Docker container
  ├─ MCP ─────────────┐
  └─ Bash / Python ───┴─► robot service
                          ├─ harness tools:
                          │   status, episode
                          │   control, Gemini
                          └─ backend
                              RoboDojo │ BEHAVIOR-1K
                              ├─ control · planner
                              └─ cameras

operator ── arm / disarm ──► <session>/operator/
                  (agent cannot write; read every tick)
coding agent (sandboxed CLI)
  ├─ MCP ─────────────┐
  └─ Bash / Python ───┴─► robot service (audit log)
                          ├─ harness tools:
                          │   status, operator, Gemini
                          └─ backend services/<robot>/
                              ├─ vendor control SDK
                              ├─ planner · URDF · cameras
                              └─ recorder (read-only)

The robot service is the only process that commands the robot. On real hardware the vendor stack stays untouched, safety is layered from a physical e-stop down to the vendor controller's limits, and the agent's sandbox has no path to the robot except the MCP tools. The adaptation guide covers the full design.

skills/ collects the universal, robot-agnostic skills every branch gives its agent: manual and autonomous control, experience memory, learning from demonstrations, transfer handoff, continual learning, multi-env and multi-agent exploration, VLM perception, camera inspection, object localization and the live stream.

• 🔌 Robot-agnostic. Skills name tools by role; a table maps each role to the tools of RoboDojo, BEHAVIOR-1K, Franka and RB-Y1.
• 🌿 Reflects each branch. A matrix links every skill to its version on each branch and says how simulation and real robots differ.
• 🦾 Ready to port. A robot-interface template holds the facts a new robot fills in.

skills-lib is a companion repository holding the task skills agents wrote on real robots. RAX is the general harness; skills-lib is what its agents learned with it.

• 🦾 One library per embodiment. Each robot has its own branch: rby1 (box, bucket and chair moves, cup stacking, whole-body hand reach, a six-step room tidy) and franka.
• 🧾 Skills with evidence. Every skill keeps its code, preconditions, checks, failure modes, run records and camera frames; it is verified only after two unattended successful runs.
• 🔁 Grows across sessions. The harness clones the robot's library into each session's workspace. Guided by the continual-learning skill, the agent reads it, runs or composes the skills that match, and merges back what it improves. Every new real robot should get one; the adaptation guide shows how.

The raw logs of the real-robot sessions are on Hugging Face: robot-session-logs. It holds 5 Franka and 7 RB-Y1 sessions. Each one keeps the agent's full transcripts, every robot tool call with its result, the robot state log and camera recordings, the operator's instructions, and the code, notes and skills the agent wrote.

A new robot needs a backend, not a dataset. The real-robot adaptation guide walks through it:

• Five design principles: wire the vendor stack without modifying it, measure every control mode before advising the agent, universal skills in a sandbox with a skill library that persists across sessions, observe on request and record continuously, and keep the harness separate from the robot.
• The backend contract, layered safety and the shared error shape.
• The characterization protocol and how to turn its numbers into skills.
• A 12-step bring-up checklist, from mapping the system to a dry agent session with a person at the e-stop.

The simulation branches build on RoboDojo, BEHAVIOR-1K, HumanoidArena, SONIC, DexJoCo, MuJoCo, NVIDIA Isaac Sim and Isaac Lab, and cuRobo. Each branch lists the exact versions it pins.

RAX's original source code and accompanying documentation are licensed under the Apache License 2.0. Copyright 2026 RAX contributors.

Third-party code, simulator software, assets, model weights and datasets retain their own licenses. See NOTICE and the relevant upstream releases.

If you find RAX useful, please cite:

@article{huang2026scaling,
  title={Scaling Online Exploration for Robot Agents},
  author={Huang, Runhan and Zhang, Xiangcheng and Chen, Haoyu and Li, Yitong and Wang, Yen-Jen and Lyu, Bohan and Ye, Haotian and Hong, Chuye and Wang, Yixuan and Su, Chloe and others},
  journal={arXiv preprint arXiv:2610.22496},
  year={2026}
}