Evo-RL

SJTU & Evo-Tech

Architecture Overview

🎯 Evo-RL Focus

Open real-world RL on two platforms: we build and release full real-world RL pipelines on SO101 and AgileX (PiPER/PiPER-X).
Open code, models, and datasets for reproducibility: we continuously release runnable offline RL assets so more people can reproduce results and apply them to real-world tasks.
Open algorithm and community co-evolution: we reproduce existing real-world RL methods, propose new methods, and keep publishing data/benchmarks to grow a collaborative open-source community.

🚀 News

[2026-03-07] Added AgileX (PiPER/PiPER-X) support for real-world RL.
[2026-02-26] First SO101 real-world RL baseline and reproducible CLI workflow are released.

🧭 Table of Contents

Getting Started	Training Pipeline	Project Info
⚡ Quick Start	4) Value Function Training	Model & Dataset
1) Installation	5) Value Inference	Community Channels
2) Hardware Setup	6) Policy Training	Affiliations
3) Data Collection	7) Closed-loop Rollout and Next Round	Citation / License

Value Visual Results

Success Case

Failure Case

Policy Rollout Visual Results

Human-in-the-Loop Visual Results

⚡ Quick Start

LeRobot-aligned foundation: we use LeRobot as the base of this codebase because its inference and data-collection logic are highly aligned with real-world RL workflows.

DexJoCo Verified Data Collection & Evaluation Path

For the current click_mouse DexJoCo benchmark reproduction, the verified path is:

start an OpenPI policy server first
run the DexJoCo-side evaluator separately
keep visual observations enabled with render_mode=rgb_array
disable saved video files with --no-record-video
use OPENPI_ACTION_CHUNK_TIMESTAMP_MODE=obs_plus_1

Two practical modes are verified:

data collection mode: save per-episode raw rollout .npz
evaluation-only mode: do not save raw rollout data, only keep the final success-rate file

1. Start the OpenPI policy server

Run this in the openpi environment.

source ~/miniconda3/etc/profile.d/conda.sh
conda activate openpi
cd /home/yangrongzheng/Evo-RL/dexjoco/openpi

export LD_LIBRARY_PATH=$CONDA_PREFIX/lib:$LD_LIBRARY_PATH
export PYTHONPATH=/home/yangrongzheng/Evo-RL/src:/home/yangrongzheng/Evo-RL/dexjoco/openpi/packages/openpi-client/src:/home/yangrongzheng/Evo-RL/dexjoco/openpi/src:$PYTHONPATH
export CUDA_VISIBLE_DEVICES=6
export XLA_PYTHON_CLIENT_PREALLOCATE=false
export XLA_PYTHON_CLIENT_MEM_FRACTION=0.2

python ./scripts/serve_policy.py \
  --port=8019 \
  policy:checkpoint \
  --policy.config=click_mouse \
  --policy.dir=/home/yangrongzheng/Evo-RL/click_mouse/pi05_dexjoco_ckpt/click_mouse

This is the verified base-policy serving setup. If you want to evaluate another checkpoint, replace only --policy.dir.

2. Verified data collection probe

Run this in the dexjoco environment. This is the verified collection path that saves raw rollout .npz files.

source ~/miniconda3/etc/profile.d/conda.sh
conda activate dexjoco
cd /home/yangrongzheng/Evo-RL/dexjoco

export LD_LIBRARY_PATH=$CONDA_PREFIX/lib:$LD_LIBRARY_PATH
export PYTHONPATH=/home/yangrongzheng/Evo-RL/src:/home/yangrongzheng/Evo-RL/dexjoco/dexjoco:/home/yangrongzheng/Evo-RL/dexjoco/openpi/packages/openpi-client/src:$PYTHONPATH
export MUJOCO_GL=egl
export CUDA_VISIBLE_DEVICES=6
export OPENPI_ACTION_CHUNK_TIMESTAMP_MODE=obs_plus_1

python -u -m dexjoco_openpi_client.cli.evaluate \
  --config=./configs/rand_obj/click_mouse.yaml \
  --seed=0 \
  --port=8019 \
  --episodes=1 \
  --render-mode=rgb_array \
  --no-record-video \
  --host=127.0.0.1 \
  --output=/home/yangrongzheng/Evo-RL/outputs/dexjoco_click_mouse_eval_probe_rgb_novideo \
  --raw-rollout-root=/home/yangrongzheng/Evo-RL/outputs/dexjoco_click_mouse_raw_probe_rgb_novideo

This generates:

success-rate summary under outputs/dexjoco_click_mouse_eval_probe_rgb_novideo
per-episode raw rollout .npz under outputs/dexjoco_click_mouse_raw_probe_rgb_novideo

3. Verified 200-episode data collection

source ~/miniconda3/etc/profile.d/conda.sh
conda activate dexjoco
cd /home/yangrongzheng/Evo-RL/dexjoco

export LD_LIBRARY_PATH=$CONDA_PREFIX/lib:$LD_LIBRARY_PATH
export PYTHONPATH=/home/yangrongzheng/Evo-RL/src:/home/yangrongzheng/Evo-RL/dexjoco/dexjoco:/home/yangrongzheng/Evo-RL/dexjoco/openpi/packages/openpi-client/src:$PYTHONPATH
export MUJOCO_GL=egl
export CUDA_VISIBLE_DEVICES=6
export OPENPI_ACTION_CHUNK_TIMESTAMP_MODE=obs_plus_1

python -u -m dexjoco_openpi_client.cli.evaluate \
  --config=./configs/rand_obj/click_mouse.yaml \
  --seed=0 \
  --port=8019 \
  --episodes=200 \
  --render-mode=rgb_array \
  --no-record-video \
  --host=127.0.0.1 \
  --output=/home/yangrongzheng/Evo-RL/outputs/dexjoco_click_mouse_eval_200_rgb_novideo \
  --raw-rollout-root=/home/yangrongzheng/Evo-RL/outputs/dexjoco_click_mouse_raw_200_rgb_novideo

After completion:

ls /home/yangrongzheng/Evo-RL/outputs/dexjoco_click_mouse_eval_200_rgb_novideo/success_rate_*
ls /home/yangrongzheng/Evo-RL/outputs/dexjoco_click_mouse_raw_200_rgb_novideo | wc -l

4. ACP-conditional evaluation (ACP=1 prompt injection)

Evaluation-time ACP is controlled by the prompt passed to the policy server.

No ACP injection (default): do not set OPENPI_DEFAULT_PROMPT when starting serve_policy.py. In this mode, the policy uses the task prompt from the evaluation config.
ACP=1 injection: set OPENPI_DEFAULT_PROMPT to the task text plus a newline and Advantage: positive before starting serve_policy.py.
Where to change the prompt:
- change the default non-ACP task prompt in dexjoco/configs/rand_obj/click_mouse.yaml
- or override the server-side prompt at runtime with OPENPI_DEFAULT_PROMPT

For click_mouse, the default non-ACP prompt in the eval config is:

Move the mouse to the purple mouse pad and click the left mouse button.

To evaluate with ACP=1 prompt injection, use the same task text plus newline-tag format as the training-side ACP transform. For click_mouse, the injected prompt should be:

Move the mouse to the purple mouse pad and click the left mouse button.
Advantage: positive

Set that exact prompt through OPENPI_DEFAULT_PROMPT before starting the server:

# Terminal 1: Start server with ACP=1 prompt
cd /home/yangrongzheng/Evo-RL/dexjoco/openpi && source ~/miniconda3/etc/profile.d/conda.sh && conda activate openpi && export LD_LIBRARY_PATH=$CONDA_PREFIX/lib:$LD_LIBRARY_PATH && export PYTHONPATH=/home/yangrongzheng/Evo-RL/src:/home/yangrongzheng/Evo-RL/dexjoco/openpi/packages/openpi-client/src:/home/yangrongzheng/Evo-RL/dexjoco/openpi/src:$PYTHONPATH && export CUDA_VISIBLE_DEVICES=6 && export XLA_PYTHON_CLIENT_PREALLOCATE=false && export XLA_PYTHON_CLIENT_MEM_FRACTION=0.2 && export OPENPI_DEFAULT_PROMPT=$'Move the mouse to the purple mouse pad and click the left mouse button.\nAdvantage: positive' && python ./scripts/serve_policy.py --port=8005 policy:checkpoint --policy.config=click_mouse --policy.dir=/home/yangrongzheng/Evo-RL/dexjoco/checkpoints/pi05_ckpts/click_mouse/click_mouse20260622/29999

# Terminal 2: Run evaluation (after server is ready)
cd /home/yangrongzheng/Evo-RL/dexjoco && source ~/miniconda3/etc/profile.d/conda.sh && conda activate dexjoco && export LD_LIBRARY_PATH=$CONDA_PREFIX/lib:$LD_LIBRARY_PATH && export PYTHONPATH=/home/yangrongzheng/Evo-RL/src:/home/yangrongzheng/Evo-RL/dexjoco/dexjoco:/home/yangrongzheng/Evo-RL/dexjoco/openpi/packages/openpi-client/src:$PYTHONPATH && export MUJOCO_GL=egl && export CUDA_VISIBLE_DEVICES=0 && export OPENPI_ACTION_CHUNK_TIMESTAMP_MODE=obs_plus_1 && python -u -m dexjoco_openpi_client.cli.evaluate --config=./configs/rand_obj/click_mouse.yaml --seed=0 --port=8005 --episodes=20 --render-mode=rgb_array --no-record-video --host=127.0.0.1 --output=/home/yangrongzheng/Evo-RL/outputs/eval_ckpt29999_acp1

This injects Advantage: positive into the task prompt, so the policy generates actions conditioned on ACP=1.

If you want to explicitly run without ACP injection, make sure OPENPI_DEFAULT_PROMPT is unset before starting the server:

unset OPENPI_DEFAULT_PROMPT

5. Evaluation-only mode (no raw rollout saved)

If you only want success/failure statistics and do not want to save rollout data, use the same evaluator path but do not pass --raw-rollout-root.

source ~/miniconda3/etc/profile.d/conda.sh
conda activate dexjoco
cd /home/yangrongzheng/Evo-RL/dexjoco

export LD_LIBRARY_PATH=$CONDA_PREFIX/lib:$LD_LIBRARY_PATH
export PYTHONPATH=/home/yangrongzheng/Evo-RL/src:/home/yangrongzheng/Evo-RL/dexjoco/dexjoco:/home/yangrongzheng/Evo-RL/dexjoco/openpi/packages/openpi-client/src:$PYTHONPATH
export MUJOCO_GL=egl
export CUDA_VISIBLE_DEVICES=6
export OPENPI_ACTION_CHUNK_TIMESTAMP_MODE=obs_plus_1

python -u -m dexjoco_openpi_client.cli.evaluate \
  --config=./configs/rand_obj/click_mouse.yaml \
  --seed=0 \
  --port=8019 \
  --episodes=200 \
  --render-mode=rgb_array \
  --no-record-video \
  --host=127.0.0.1 \
  --output=/home/yangrongzheng/Evo-RL/outputs/dexjoco_click_mouse_eval_nosave_200

This mode only keeps the final success-rate file under the output directory, for example:

ls /home/yangrongzheng/Evo-RL/outputs/dexjoco_click_mouse_eval_nosave_200/success_rate_*

Notes

The verified path uses render_mode=rgb_array; do not switch this to none if you want the previously validated collection behavior.
--no-record-video disables writing video files to disk, but still keeps image rendering active for policy inference.
The evaluator prompt comes from dexjoco/configs/rand_obj/click_mouse.yaml unless the server is started with OPENPI_DEFAULT_PROMPT, in which case the environment-variable prompt overrides the config prompt.
recollect_click_mouse_dataset_only.sh is a wrapper around the verified evaluator + raw-rollout-export path; it does not start the policy server for you, so the server must already be running on the same port.

4.1 Faithful EvoRL ACP-positive rollout collection

If you want rollout collection to reuse EvoRL's own ACP inference implementation instead of inventing a separate DexJoCo-only path, use the ACP-positive collection script below.

This path reuses the same task/prompt tagging helper used by EvoRL HIL inference:

src/lerobot/scripts/recording_hil.py
src/lerobot/rl/acp_tags.py

Specifically, rollout prompt injection is implemented by calling:

build_acp_tagged_task(task, is_positive=True)

The DexJoCo rollout entrypoint now reuses that exact helper when OPENPI_FORCE_ACP_POSITIVE=1 is set.

To collect 200 episodes with the trained 9999 checkpoint and save the raw data under /share_data/yangrongzheng/:

nohup bash /home/yangrongzheng/Evo-RL/dexjoco/scripts/recollect_click_mouse_9999_acp_positive_to_share.sh > /tmp/recollect_click_mouse_9999_acp1.stdout 2>&1 &

Defaults used by this script:

policy checkpoint: /home/yangrongzheng/Evo-RL/dexjoco/checkpoints/pi05_ckpts/click_mouse/click_mouse20260615/9999
GPU: 6
port: 8019
episodes: 200
ACP rollout mode: OPENPI_FORCE_ACP_POSITIVE=1
timestamp mode: OPENPI_ACTION_CHUNK_TIMESTAMP_MODE=obs_plus_1

Outputs:

raw rollout data: /share_data/yangrongzheng/dexjoco_click_mouse_raw_200_rgb_novideo_img_acp1_9999
rollout summary dir: /share_data/yangrongzheng/dexjoco_click_mouse_eval_200_rgb_novideo_img_acp1_9999
converted LeRobot dataset: /share_data/yangrongzheng/click_mouse_value_train_200_img_acp1_9999

Useful logs:

tail -f /tmp/recollect_click_mouse_9999_acp1.stdout
tail -f /tmp/click_mouse_9999_acp1_server.log
tail -f /tmp/click_mouse_9999_acp1_collect.log

5. Value model training on the collected 200 episodes

The verified 200-episode image rollout dataset and converted LeRobot dataset are currently located at:

raw rollout data: /share_data/yangrongzheng/dexjoco_click_mouse_raw_200_rgb_novideo_img
rollout summary dir: /share_data/yangrongzheng/dexjoco_click_mouse_eval_200_rgb_novideo_img
converted LeRobot dataset: /share_data/yangrongzheng/click_mouse_value_train_200_img

To train the value model for 10k steps and save every 2k steps:

nohup env GPU=5 bash /home/yangrongzheng/Evo-RL/dexjoco/scripts/train_value_pi05_200_with_periodic_eval.sh > /tmp/train_value_pistar06_200_10k_gpu5.log 2>&1 &

Defaults used by this script:

raw input: /share_data/yangrongzheng/dexjoco_click_mouse_raw_200_rgb_novideo_img
LeRobot dataset: /share_data/yangrongzheng/click_mouse_value_train_200_img
output dir: /share_data/yangrongzheng/value_train/click_mouse_pistar06_200_10k
training steps: 10000
checkpoint frequency: 2000

The script automatically converts raw .npz into LeRobot format if meta/info.json is missing.

Monitor training with:

tail -f /tmp/train_value_pistar06_200_10k_gpu5.log

6. OpenPI training smoke with ACP sidecar

For the aligned JAX/OpenPI training path, the current smoke training entry is:

nohup env GPU=3 bash /home/yangrongzheng/Evo-RL/dexjoco/scripts/train_click_mouse_openpi_acp_smoke.sh > /tmp/train_click_mouse_openpi_acp_smoke_gpu3.log 2>&1 &

For click_mouse, do not rely on the converted LeRobot parquet dataset under /share_data/yangrongzheng/click_mouse/... for OpenPI training. On the current environment, that path can fail during parquet loading with PyArrow nested-chunk conversion errors.

The failure usually looks like:

pyarrow.lib.ArrowNotImplementedError: Nested data conversions not implemented for chunked array outputs

This happens because the converted LeRobot parquet for click_mouse contains nested columns (for example fixed-size-list actions/states and image structs), and the current PyArrow + datasets stack can fail when those nested fields are materialized through chunked parquet reads. We first tried to patch the parquet fallback path in src/lerobot/datasets/utils.py, but the more reliable solution for OpenPI training on this environment is to bypass parquet entirely and train from the verified raw rollout .npz files via OPENPI_RAW_NPZ_ROOT.

Use the verified raw rollout NPZ path instead:

cd /home/yangrongzheng/Evo-RL/dexjoco/openpi && \
source ~/miniconda3/etc/profile.d/conda.sh && \
conda activate openpi && \
export HF_DATASETS_CACHE=/tmp/hf_datasets_cache && \
mkdir -p /tmp/hf_datasets_cache && \
export PYTHONPATH=/home/yangrongzheng/Evo-RL/src:$PYTHONPATH && \
export OPENPI_RAW_NPZ_ROOT=/share_data/yangrongzheng/dexjoco_click_mouse_raw_200_rgb_novideo_img && \
CUDA_VISIBLE_DEVICES=7 \
LEROBOT_ACP_SIDECAR_NPZ=/tmp/click_mouse_value_train_200_img_trainsafe_v2_acp_sidecar.npz \
LEROBOT_ACP_DROPOUT=0.3 \
nohup python scripts/train.py click_mouse --overwrite > /tmp/train_large2.log 2>&1 &

Do not use the old /tmp/acp_sidecar.npz file here. The verified ACP sidecar for this training pipeline is /tmp/click_mouse_value_train_200_img_trainsafe_v2_acp_sidecar.npz, which contains 105920 aligned entries with acp_indicator/value/advantage and matches the raw rollout NPZ training set.

To resume the same day’s run, keep the same OPENPI_RAW_NPZ_ROOT export and switch to --resume:

cd /home/yangrongzheng/Evo-RL/dexjoco/openpi && \
source ~/miniconda3/etc/profile.d/conda.sh && \
conda activate openpi && \
export HF_DATASETS_CACHE=/tmp/hf_datasets_cache && \
mkdir -p /tmp/hf_datasets_cache && \
export PYTHONPATH=/home/yangrongzheng/Evo-RL/src:$PYTHONPATH && \
export OPENPI_RAW_NPZ_ROOT=/share_data/yangrongzheng/dexjoco_click_mouse_raw_200_rgb_novideo_img && \
CUDA_VISIBLE_DEVICES=7 \
LEROBOT_ACP_SIDECAR_NPZ=/tmp/click_mouse_value_train_200_img_trainsafe_v2_acp_sidecar.npz \
LEROBOT_ACP_DROPOUT=0.3 \
nohup python scripts/train.py click_mouse --resume > /tmp/train_large_resume.log 2>&1 &

If the run has already reached the default 30000 training steps, --resume alone will restore the latest checkpoint and then exit immediately. To continue training the same run beyond 30k steps, increase --num-train-steps. For example, to continue the 20260630 run from 29999 to 60000 total steps:

cd /home/yangrongzheng/Evo-RL/dexjoco/openpi && \
source ~/miniconda3/etc/profile.d/conda.sh && \
conda activate openpi && \
export HF_DATASETS_CACHE=/tmp/hf_datasets_cache && \
mkdir -p /tmp/hf_datasets_cache && \
export PYTHONPATH=/home/yangrongzheng/Evo-RL/src:$PYTHONPATH && \
export OPENPI_RAW_NPZ_ROOT=/share_data/yangrongzheng/dexjoco_click_mouse_raw_200_rgb_novideo_img && \
CUDA_VISIBLE_DEVICES=5 \
LEROBOT_ACP_SIDECAR_NPZ=/tmp/click_mouse_value_train_200_img_trainsafe_v2_acp_sidecar.npz \
LEROBOT_ACP_DROPOUT=0.3 \
nohup python scripts/train.py click_mouse --resume --num-train-steps 60000 > /tmp/train_large_resume.log 2>&1 &

Monitor training with:

tail -f /tmp/train_large2.log
tail -f /tmp/train_large_resume.log

The training log now reports both model-step time and data-loading time:

train_step_s: JAX forward/backward/update time
data_loading_s: data pipeline wait time

For the verified click_mouse raw-npz path, the main bottleneck is usually data_loading_s rather than train_step_s.

6.1 OpenPI evaluation with full ACP=1 prompt injection

For evaluation, ACP=1 prompt injection must match the same string format used by training-side ACP tagging: the task text followed by a newline and Advantage: positive.

The exact prompt is:

Move the mouse to the purple mouse pad and click the left mouse button.
Advantage: positive

Start the server with that prompt injected through OPENPI_DEFAULT_PROMPT:

cd /home/yangrongzheng/Evo-RL/dexjoco/openpi && \
source ~/miniconda3/etc/profile.d/conda.sh && \
conda activate openpi && \
export LD_LIBRARY_PATH=$CONDA_PREFIX/lib:$LD_LIBRARY_PATH && \
export PYTHONPATH=/home/yangrongzheng/Evo-RL/src:/home/yangrongzheng/Evo-RL/dexjoco/openpi/packages/openpi-client/src:/home/yangrongzheng/Evo-RL/dexjoco/openpi/src:$PYTHONPATH && \
export CUDA_VISIBLE_DEVICES=5 && \
export XLA_PYTHON_CLIENT_PREALLOCATE=false && \
export XLA_PYTHON_CLIENT_MEM_FRACTION=0.2 && \
export OPENPI_DEFAULT_PROMPT=$'Move the mouse to the purple mouse pad and click the left mouse button.\nAdvantage: positive' && \
nohup python ./scripts/serve_policy.py --port=8010 policy:checkpoint --policy.config=click_mouse --policy.dir=/home/yangrongzheng/Evo-RL/dexjoco/checkpoints/pi05_ckpts/click_mouse/click_mouse20260630/59999 > /tmp/click_mouse_ckpt59999_server_acp1.log 2>&1 &

Then run evaluation without saving videos:

cd /home/yangrongzheng/Evo-RL/dexjoco && \
source ~/miniconda3/etc/profile.d/conda.sh && \
conda activate dexjoco && \
export LD_LIBRARY_PATH=$CONDA_PREFIX/lib:$LD_LIBRARY_PATH && \
export PYTHONPATH=/home/yangrongzheng/Evo-RL/src:/home/yangrongzheng/Evo-RL/dexjoco/dexjoco:/home/yangrongzheng/Evo-RL/dexjoco/openpi/packages/openpi-client/src:$PYTHONPATH && \
export MUJOCO_GL=egl && \
export CUDA_VISIBLE_DEVICES=0 && \
export OPENPI_ACTION_CHUNK_TIMESTAMP_MODE=obs_plus_1 && \
nohup python -u -m dexjoco_openpi_client.cli.evaluate \
  --config=./configs/rand_obj/click_mouse.yaml \
  --seed=0 \
  --port=8010 \
  --episodes=200 \
  --render-mode=rgb_array \
  --no-record-video \
  --host=127.0.0.1 \
  --output=/home/yangrongzheng/Evo-RL/outputs/dexjoco_click_mouse_eval_ckpt59999_200ep_acp1 \
  > /tmp/eval_click_mouse_ckpt59999_200ep_acp1.log 2>&1 &

Useful logs:

tail -f /tmp/click_mouse_ckpt59999_server_acp1.log
tail -f /tmp/eval_click_mouse_ckpt59999_200ep_acp1.log

This ACP=1 evaluation setup is the verified path that produced 75.5% success rate on 200 episodes for ckpt59999.

Monitor it with:

tail -f /tmp/train_click_mouse_openpi_acp_smoke_gpu3.log

1) Installation

git clone https://github.com/MINT-SJTU/Evo-RL.git
cd Evo-RL
conda create -y -n evo-rl python=3.10
conda activate evo-rl
pip install -e .

For setup details and platform-specific dependencies, follow the official LeRobot configuration guide.

2) Hardware Setup

SO Series (SO100/SO101)

For SO-series setup, please follow the official tutorial in detail and complete all installation and configuration steps there before continuing. The examples below use SO101 as the reference configuration.

Device path recommendation

Recommended path strategy:

Robot serial: use /dev/serial/by-id/ (stable across reboots).
Cameras: prefer /dev/v4l/by-id/; if IDs are not unique, use /dev/v4l/by-path/.
In examples below: robot ports use by-id, camera paths use by-path.

You can inspect available stable paths with:

ls -l /dev/serial/by-id/
ls -l /dev/v4l/by-id/
ls -l /dev/v4l/by-path/

For single-arm users, no major changes are required. After setup, run the command below to verify your system is ready for the next stage:

lerobot-teleoperate \
  --robot.type=so101_follower \
  --robot.port=/dev/serial/by-id/<SO101_FOLLOWER_PORT> \
  --robot.id=my_so101_follower \
  --teleop.type=so101_leader \
  --teleop.port=/dev/serial/by-id/<SO101_LEADER_PORT> \
  --teleop.id=my_so101_leader

For dual-arm users, we recommend mirroring the mechanical parts corresponding to servos 4/5/6 on the left leader and left follower arms, which usually provides a more natural bimanual operation feel.

Before running the dual-arm command, make sure calibration files exist under ~/.cache/huggingface/lerobot/calibration/ like:

calibration/
├── robots
│   └── so_follower
│       ├── bi_so101_follower_left.json
│       └── bi_so101_follower_right.json
└── teleoperators
    └── so_leader
        ├── bi_so101_leader_left.json
        └── bi_so101_leader_right.json

This layout is slightly different from single-arm setup.

Then run this command to verify dual-arm setup:

lerobot-teleoperate \
  --robot.type=bi_so_follower \
  --robot.left_arm_config.port=/dev/serial/by-id/<LEFT_FOLLOWER_PORT> \
  --robot.right_arm_config.port=/dev/serial/by-id/<RIGHT_FOLLOWER_PORT> \
  --robot.id=bi_so101_follower \
  --teleop.type=bi_so_leader \
  --teleop.left_arm_config.port=/dev/serial/by-id/<LEFT_LEADER_PORT> \
  --teleop.right_arm_config.port=/dev/serial/by-id/<RIGHT_LEADER_PORT> \
  --teleop.id=bi_so101_leader

Camera configuration

Before data collection, validate camera mapping first.

Check whether each camera supports your target setting (for example, 640x480 @ 30):

v4l2-ctl -d /dev/v4l/by-path/<CAM_PATH> --list-formats-ext

Single-arm camera check (example):

lerobot-teleoperate \
  --robot.type=so101_follower \
  --robot.port=/dev/serial/by-id/<SO101_FOLLOWER_PORT> \
  --robot.id=my_so101_follower \
  --robot.cameras='{ front: {type: opencv, index_or_path: "/dev/v4l/by-path/<FRONT_CAM>", width: 640, height: 480, fps: 30}}' \
  --teleop.type=so101_leader \
  --teleop.port=/dev/serial/by-id/<SO101_LEADER_PORT> \
  --teleop.id=my_so101_leader \
  --display_data=true

Dual-arm camera check (example):

lerobot-teleoperate \
  --robot.type=bi_so_follower \
  --robot.left_arm_config.port=/dev/serial/by-id/<LEFT_FOLLOWER_PORT> \
  --robot.right_arm_config.port=/dev/serial/by-id/<RIGHT_FOLLOWER_PORT> \
  --robot.id=my_bi_so101_follower \
  --robot.left_arm_config.cameras='{ wrist: {type: opencv, index_or_path: "/dev/v4l/by-path/<LEFT_WRIST_CAM_PATH>", width: 640, height: 480, fps: 30}}' \
  --robot.right_arm_config.cameras='{ wrist: {type: opencv, index_or_path: "/dev/v4l/by-path/<RIGHT_WRIST_CAM_PATH>", width: 640, height: 480, fps: 30}, front: {type: opencv, index_or_path: "/dev/v4l/by-path/<FRONT_CAM_PATH>", width: 640, height: 480, fps: 30}}' \
  --teleop.type=bi_so_leader \
  --teleop.left_arm_config.port=/dev/serial/by-id/<LEFT_LEADER_PORT> \
  --teleop.right_arm_config.port=/dev/serial/by-id/<RIGHT_LEADER_PORT> \
  --teleop.id=my_bi_so101_leader \
  --display_data=true

For dual-arm camera mapping, it is fine to attach front under either the left-arm or right-arm camera config. If you use more camera views, place them under either the left or right arm camera config as well.

If needed, you can also use temporary device paths (for example /dev/ttyACM* and /dev/video*) during initial debugging.

AgileX (PiPER/PiPER-X)

PiPER arms in master/teaching mode cannot receive external control commands, so all arms must be configured to follower/motion-output mode (0xFC), and firmware must be version 1.8.5 or above.

For PiPER-series robots, make sure Git LFS assets are pulled before running teleoperation:

git lfs pull --include="src/lerobot/assets/piper_description/**,src/lerobot/assets/piper_x_description/**" --exclude="*"
git lfs checkout src/lerobot/assets/piper_description src/lerobot/assets/piper_x_description

For PiPER setup, PiPER uses CAN interfaces instead of serial ports. So first run lerobot-setup-can to confirm CAN interfaces are available:

lerobot-setup-can --mode=setup --interfaces=<LEFT_FOLLOWER_CAN_PORT>,<LEFT_LEADER_CAN_PORT>,<RIGHT_FOLLOWER_CAN_PORT>,<RIGHT_LEADER_CAN_PORT>

For single-arm users, run the command below to verify the system is ready:

lerobot-teleoperate \
  --robot.type=piperx_follower \
  --robot.port=<FOLLOWER_CAN_PORT> \
  --robot.id=my_piperx_follower \
  --robot.require_calibration=false \
  --teleop.type=piperx_leader \
  --teleop.port=<LEADER_CAN_PORT> \
  --teleop.id=my_piperx_leader \
  --teleop.require_calibration=false

For bimanual users, run this command to verify dual-arm teleoperation:

lerobot-teleoperate \
  --robot.type=bi_piperx_follower \
  --robot.id=my_bi_piperx_follower \
  --robot.left_arm_config.port=<LEFT_FOLLOWER_CAN_PORT> \
  --robot.right_arm_config.port=<RIGHT_FOLLOWER_CAN_PORT> \
  --robot.left_arm_config.require_calibration=false \
  --robot.right_arm_config.require_calibration=false \
  --teleop.type=bi_piperx_leader \
  --teleop.id=my_bi_piperx_leader \
  --teleop.left_arm_config.port=<LEFT_LEADER_CAN_PORT> \
  --teleop.right_arm_config.port=<RIGHT_LEADER_CAN_PORT> \
  --teleop.left_arm_config.require_calibration=false \
  --teleop.right_arm_config.require_calibration=false

For PiPER (non-X), replace bi_piperx_follower/bi_piperx_leader with bi_piper_follower/bi_piper_leader.

3) Data Collection

Collect rollout data with lerobot-human-inloop-record.

SO Series (SO100/SO101)

Bimanual template:

lerobot-human-inloop-record \
  --robot.type=bi_so_follower \
  --robot.left_arm_config.port=/dev/serial/by-id/<LEFT_FOLLOWER_PORT> \
  --robot.right_arm_config.port=/dev/serial/by-id/<RIGHT_FOLLOWER_PORT> \
  --robot.id=my_bi_so101_follower \
  --robot.left_arm_config.cameras='{ wrist: {type: opencv, index_or_path: "/dev/v4l/by-path/<LEFT_WRIST_CAM_PATH>", width: 640, height: 480, fps: 30, fourcc: "MJPG"}}' \
  --robot.right_arm_config.cameras='{ wrist: {type: opencv, index_or_path: "/dev/v4l/by-path/<RIGHT_WRIST_CAM_PATH>", width: 640, height: 480, fps: 30, fourcc: "MJPG"}, front: {type: intelrealsense, serial_number_or_name: "<REALSENSE_SN>", width: 640, height: 480, fps: 30, warmup_s: 2}}' \
  --teleop.type=bi_so_leader \
  --teleop.left_arm_config.port=/dev/serial/by-id/<LEFT_LEADER_PORT> \
  --teleop.right_arm_config.port=/dev/serial/by-id/<RIGHT_LEADER_PORT> \
  --teleop.id=my_bi_so101_leader \
  --dataset.repo_id=<HF_USERNAME_OR_ORG>/<DATASET_NAME> \
  --dataset.single_task="<YOUR_TASK_DESCRIPTION>" \
  --dataset.num_episodes=<NUM_EPISODES> \
  --dataset.episode_time_s=<EPISODE_SECONDS> \
  --dataset.reset_time_s=<RESET_SECONDS> \
  --dataset.push_to_hub=true \
  --display_data=true

Recommendation: use fourcc: "MJPG" for OpenCV and warmup_s for RealSense. In this example front uses RealSense, but you can switch it to OpenCV with the same structure.

AgileX (PiPER/PiPER-X)

Bimanual template (left/right, PiPER-X example):

lerobot-human-inloop-record \
  --robot.type=bi_piperx_follower \
  --robot.id=my_bi_piperx_follower \
  --robot.left_arm_config.port=<LEFT_FOLLOWER_CAN_PORT> \
  --robot.right_arm_config.port=<RIGHT_FOLLOWER_CAN_PORT> \
  --robot.left_arm_config.require_calibration=false \
  --robot.right_arm_config.require_calibration=false \
  --teleop.type=bi_piperx_leader \
  --teleop.id=my_bi_piperx_leader \
  --teleop.left_arm_config.port=<LEFT_LEADER_CAN_PORT> \
  --teleop.right_arm_config.port=<RIGHT_LEADER_CAN_PORT> \
  --teleop.left_arm_config.require_calibration=false \
  --teleop.right_arm_config.require_calibration=false \
  --dataset.repo_id=<HF_USERNAME_OR_ORG>/<DATASET_NAME> \
  --dataset.single_task="<YOUR_TASK_DESCRIPTION>" \
  --dataset.num_episodes=<NUM_EPISODES> \
  --dataset.episode_time_s=<EPISODE_SECONDS> \
  --dataset.reset_time_s=<RESET_SECONDS> \
  --dataset.push_to_hub=true \
  --display_data=true

Hotkeys:

i: toggle intervention mode (policy <-> teleop takeover)
s: mark success and end current episode
f: mark failure and end current episode
Right Arrow: end the current loop early
Left Arrow: end early and re-record the current episode
Esc: stop the recording session

Quick quality check:

lerobot-dataset-report --dataset <HF_USERNAME_OR_ORG>/<DATASET_NAME>

This prints: dataset meta, totals, episode-length stats/histogram, success/intervention metrics, task list, and full feature schema.

4) Value Function Training

Train the value function on the current dataset. Current default: Pi*0.6 (--value.type=pistar06).

Single-GPU template:

lerobot-value-train \
  --dataset.repo_id=<HF_USERNAME_OR_ORG>/<DATASET_NAME> \
  --value.type=pistar06 \
  --value.dtype=bfloat16 \
  --value.push_to_hub=true \
  --value.repo_id=<HF_USERNAME_OR_ORG>/<VALUE_MODEL_REPO> \
  --batch_size=64 \
  --output_dir=outputs/value_train/<RUN_NAME> \
  --job_name=<RUN_NAME> \
  --wandb.enable=true

Multi-GPU template:

CUDA_VISIBLE_DEVICES=<GPU_ID_LIST> accelerate launch \
  --multi_gpu \
  --num_processes=<NUM_GPUS> \
  --mixed_precision=bf16 \
  $(which lerobot-value-train) \
  --batch_size=32/<NUM_GPUS> \
  <VALUE_TRAIN_ARGS>

To plug in a different value function, minimal path in this repo:

Add src/lerobot/values/<your_value>/configuration_<your_value>.py with @PreTrainedConfig.register_subclass("<your_value>").
Add src/lerobot/values/<your_value>/modeling_<your_value>.py with <YourValue>Policy(PreTrainedPolicy) (implement at least forward, predict_value, and build_training_raw_batch_hook for lerobot-value-train).
Add src/lerobot/values/<your_value>/processor_<your_value>.py with make_<your_value>_pre_post_processors(...).
Remove/replace the current pistar06-only type checks in src/lerobot/configs/value_train.py and src/lerobot/scripts/lerobot_value_infer.py.

5) Value Inference

Infer value signals and write value/advantage/indicator back to the dataset:

value: estimated return-to-go of the current frame.
advantage: relative improvement signal (higher means better-than-baseline trajectory quality).
indicator: binarized training tag derived from advantage.

Single-GPU template:

lerobot-value-infer \
  --dataset.repo_id=<HF_USERNAME_OR_ORG>/<DATASET_NAME> \
  --inference.checkpoint_path=outputs/value_train/<RUN_NAME> \
  --runtime.device=cuda \
  --runtime.batch_size=64 \
  --acp.enable=true \
  --acp.n_step=50 \
  --acp.positive_ratio=0.3 \
  --acp.value_field=complementary_info.value_<TAG> \
  --acp.advantage_field=complementary_info.advantage_<TAG> \
  --acp.indicator_field=complementary_info.acp_indicator_<TAG> \
  --output_dir=outputs/value_infer/<RUN_NAME> \
  --job_name=<RUN_NAME>.infer

Multi-GPU template:

CUDA_VISIBLE_DEVICES=<GPU_ID_LIST> accelerate launch \
  --multi_gpu \
  --num_processes=<NUM_GPUS> \
  --mixed_precision=bf16 \
  $(which lerobot-value-infer) \
  <VALUE_INFER_ARGS>

Parameter notes:

--acp.n_step: n-step advantage horizon.
--acp.positive_ratio: positive label ratio after advantage binarization (e.g., 0.3 = top 30% per task).

Expected new columns:

complementary_info.value_<TAG>
complementary_info.advantage_<TAG>
complementary_info.acp_indicator_<TAG>

These columns are written back to the original dataset specified by --dataset.repo_id.

6) Policy Training

Train the policy with advantage-conditioned tags. Policy requirement: it must support text/task input, because Advantage-Conditioned tags are injected into task text.

Single-GPU template:

lerobot-train \
  --dataset.repo_id=<HF_USERNAME_OR_ORG>/<DATASET_NAME> \
  --policy.type=<POLICY_TYPE> \
  --policy.pretrained_path=<POLICY_PRETRAINED_PATH> \
  --policy.device=cuda \
  --policy.dtype=bfloat16 \
  --batch_size=32 \
  --steps=30000 \
  --acp.enable=true \
  --acp.indicator_field=complementary_info.acp_indicator_<TAG> \
  --acp.indicator_dropout_prob=0.3 \
  --output_dir=outputs/train/<RUN_NAME> \
  --job_name=<RUN_NAME> \
  --wandb.enable=true \
  --policy.push_to_hub=true \
  --policy.repo_id=<HF_USERNAME_OR_ORG>/<POLICY_REPO>

Note: if your local LeRobot parquet contains nested image structs that fail in lerobot-train, first export a train-safe dataset (for example with dexjoco/scripts/raw_npz_to_lerobot_trainsafe.py) and point --dataset.root / --dataset.repo_id to that train-safe directory instead of the original dataset.

--acp.indicator_dropout_prob controls tag drop rate in task text; 0.3 helps learn both tagged and untagged conditions.

Important checks:

--acp.indicator_field must exist in the dataset and be binary (0/1).

Multi-GPU template:

CUDA_VISIBLE_DEVICES=<GPU_ID_LIST> accelerate launch \
  --multi_gpu \
  --num_processes=<NUM_GPUS> \
  --mixed_precision=bf16 \
  $(which lerobot-train) \
  --batch_size=32/<NUM_GPUS> \
  <POLICY_TRAIN_ARGS>

7) Closed-loop Rollout and Next Round

Deploy the trained policy in human-in-loop mode and collect the next dataset round:

lerobot-human-inloop-record \
  --robot.type=bi_so_follower \
  --robot.left_arm_config.port=/dev/serial/by-id/<LEFT_FOLLOWER_PORT> \
  --robot.right_arm_config.port=/dev/serial/by-id/<RIGHT_FOLLOWER_PORT> \
  --robot.id=my_bi_so101_follower \
  --robot.left_arm_config.cameras='{ wrist: {type: opencv, index_or_path: "/dev/v4l/by-path/<LEFT_WRIST_CAM_PATH>", width: 640, height: 480, fps: 30, fourcc: "MJPG"}}' \
  --robot.right_arm_config.cameras='{ wrist: {type: opencv, index_or_path: "/dev/v4l/by-path/<RIGHT_WRIST_CAM_PATH>", width: 640, height: 480, fps: 30, fourcc: "MJPG"}, front: {type: intelrealsense, serial_number_or_name: "<REALSENSE_SN>", width: 640, height: 480, fps: 30, warmup_s: 2}}' \
  --teleop.type=bi_so_leader \
  --teleop.left_arm_config.port=/dev/serial/by-id/<LEFT_LEADER_PORT> \
  --teleop.right_arm_config.port=/dev/serial/by-id/<RIGHT_LEADER_PORT> \
  --teleop.id=my_bi_so101_leader \
  --dataset.repo_id=<HF_USERNAME_OR_ORG>/<DATASET_NAME_NEXT_ROUND> \
  --dataset.single_task="<YOUR_TASK_DESCRIPTION>" \
  --dataset.num_episodes=<NUM_EPISODES> \
  --dataset.episode_time_s=<EPISODE_SECONDS> \
  --dataset.reset_time_s=<RESET_SECONDS> \
  --dataset.push_to_hub=true \
  --display_data=true \
  --policy.path=<POLICY_CHECKPOINT_OR_HUB_ID> \
  --resume=true

Dataset continuation options:

Append in place: keep --resume=true and continue recording into the same dataset.
Merge multiple rounds: use the official dataset editor to merge separate datasets.

lerobot-edit-dataset \
  --repo_id=<HF_USERNAME_OR_ORG>/<MERGED_DATASET_NAME> \
  --operation.type=merge \
  --operation.repo_ids="['<HF_USERNAME_OR_ORG>/<DATASET_ROUND_1>','<HF_USERNAME_OR_ORG>/<DATASET_ROUND_2>']"

Additional data attributes vs default lerobot-record behavior:

complementary_info.policy_action: policy output action at each step.
complementary_info.is_intervention: whether current step is in intervention.
complementary_info.state: intervention state-machine state.
complementary_info.collector_policy_id: step-level action source ID (human or policy ID).
Episode metadata episode_success: success/failure label saved per episode.

Iterative training loop (abstract):

[Multi-task demonstration data pool]
        |
        v
[Offline RL pretraining for a vision-language-action policy]
        |
        v
[Task-specific initialization / fine-tuning from demonstrations]
        |
        v
|---- Iteration k = 1..K -------------------------------------|
| 1) Deploy current policy π_k and collect new rollout data   |
| 2) Merge into data pool: D <- D U new_data                  |
| 3) Train value function on D                                |
| 4) Infer advantage and binarize into indicator tags         |
| 5) Train advantage-conditioned policy to get π_{k+1}        |
|-------------------------------------------------------------|
        |
        v
[Stronger policy with improved success rate and throughput]

Model & Dataset

Hugging Face model release: coming soon
Hugging Face dataset release: coming soon
Once published, this section will pin canonical repos and exact version tags.

Community Channels

WeChat official post: Coming Soon
Documentation: docs/README.md
GitHub Issues: Create an issue
Email: business@evomind-tech.com
WeChat group QR code:

So101 Supplier WeChat Contact, So101 设备提供商:

Affiliations

Citation

@misc{evorl2026,
  title        = {Evo-RL: Towards Iterative Policy Improvement in Real-World Offline RL},
  author       = {Evo-RL Contributors},
  year         = {2026},
  howpublished = {\url{https://github.com/MINT-SJTU/Evo-RL}}
}

License

Apache-2.0. See LICENSE.

Name		Name	Last commit message	Last commit date
Latest commit History 66 Commits
.github		.github
benchmarks/video		benchmarks/video
dexjoco		dexjoco
docker		docker
docs		docs
examples		examples
pi05_model_dexjoco_pick/pi05_dexjoco_ckpt/pick_bucket/params/ocdbt.process_0/d		pi05_model_dexjoco_pick/pi05_dexjoco_ckpt/pick_bucket/params/ocdbt.process_0/d
src/lerobot		src/lerobot
tests		tests
.dockerignore		.dockerignore
.gitattributes		.gitattributes
.gitignore		.gitignore
.pre-commit-config.yaml		.pre-commit-config.yaml
CODE_OF_CONDUCT.md		CODE_OF_CONDUCT.md
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
MANIFEST.in		MANIFEST.in
Makefile		Makefile
README.md		README.md
SECURITY.md		SECURITY.md
docs-requirements.txt		docs-requirements.txt
eval_click_mouse_ckpt10000_acp1.sh		eval_click_mouse_ckpt10000_acp1.sh
eval_click_mouse_no_save.sh		eval_click_mouse_no_save.sh
nohup.out		nohup.out
pyproject.toml		pyproject.toml
requirements-macos.txt		requirements-macos.txt
requirements-ubuntu.txt		requirements-ubuntu.txt
requirements.in		requirements.in
setup.py		setup.py
simple_eval_click_mouse.py		simple_eval_click_mouse.py
train_pi05_acp_small.sh		train_pi05_acp_small.sh
train_pi05_acp_v3.sh		train_pi05_acp_v3.sh

Folders and files

Latest commit

History

Repository files navigation

Evo-RL

🎯 Evo-RL Focus

🚀 News

🧭 Table of Contents

⚡ Quick Start

DexJoCo Verified Data Collection & Evaluation Path

1. Start the OpenPI policy server

2. Verified data collection probe

3. Verified 200-episode data collection

4. ACP-conditional evaluation (ACP=1 prompt injection)

5. Evaluation-only mode (no raw rollout saved)

Notes

4.1 Faithful EvoRL ACP-positive rollout collection

5. Value model training on the collected 200 episodes

6. OpenPI training smoke with ACP sidecar

6.1 OpenPI evaluation with full ACP=1 prompt injection

1) Installation

2) Hardware Setup

SO Series (SO100/SO101)

Device path recommendation

Camera configuration

AgileX (PiPER/PiPER-X)

3) Data Collection

SO Series (SO100/SO101)

AgileX (PiPER/PiPER-X)

4) Value Function Training

5) Value Inference

6) Policy Training

7) Closed-loop Rollout and Next Round

Model & Dataset

Community Channels

Affiliations

Citation

License

Star History

About

Resources

Code of conduct

Contributing

Security policy

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages