RL-Robotics

Config-first quadruped locomotion RL research pipeline.

Domain: velocity tracking, terrain traversal, and gait stability on Unitree Go2 / ANYmal-class platforms.

Goal: reproducible, audit-friendly closed-loop robot learning — simulation-first, with an optional real-robot escalation path. Every run is reconstructible from git commit + frozen config + artifact tree.

Overview

Closed-loop locomotion learning here means:

observation → policy → action → physics/robot backend → measurement → reward → update

The pipeline enforces research discipline by design:

Principle	What it means in practice
Config-first	One canonical `PipelineConfig`; TOML for CI and shared runs
Immutable run identity	`run_name`, `seed`, `git_commit`, `config_hash` stamped on every summary
One artifact contract	Predictable tree under `runs/{run_name}/` — no ad-hoc filenames
Provenance on metrics	Each metric tagged with `metric_sources` (sim, real, external, human)
Explicit learning scope	Summaries declare what was trained vs fixed (URDF, reward, safety limits)
Claim boundaries	Valid and invalid claims embedded per evidence tier
Safety by default	Guardrails, estop semantics, no silent fallback on real hardware
Deterministic replay	Hash-chained episode manifests for audit
Learned + baseline policies	Clipped PPO (PyTorch) and PD velocity baseline via `training.policy_mode`

Layers: configuration → orchestration → learning → measurement → analysis → artifacts

Optional: pip install -e ".[plot,track]" for figures and W&B. MuJoCo and (for PPO) Torch come with .[dev].

Real hardware: optional Rust safety supervisor (quad-bridge/) — see docs/integrations.md.

See docs/architecture.md for the full topology.

Who this is for

Role	Start here
Research engineer	Quick start → Common workflows
Paper author	Paper workflow → `quad bundle`
Reviewer / auditor	Reproducibility & audit
Hardware operator	Safety → `real_lab_gated` preset
Sim integrator	Measurement backends

System requirements

Resource	Linux (recommended)	macOS	Windows
OS	Ubuntu 22.04+ x86_64	macOS 13+ (Apple Silicon or Intel)	Windows 10/11
Python	3.11 – 3.12	3.11 – 3.12	3.11 – 3.12
Git	required	required	required
RAM	8 GB (CI smoke) / 32 GB (training)	8 GB minimum	8 GB minimum
GPU	Recommended for PPO (`training.device=auto`)	Optional	Optional
Real robot	ROS 2 + `ros2_control`	Not supported	Not supported

Platform note: quad doctor may show platform_linux_recommended: FAIL on macOS/Windows. That is expected — sim development works; production training and hardware integration target Linux.

Quick start

Requires Python 3.11+ and git. After install, quad is available as a console entry point inside the activated virtual environment.

Linux (recommended)

# Ubuntu/Debian prerequisites
sudo apt update && sudo apt install -y python3 python3-venv python3-pip git

git clone https://github.com/Legendarylibr/RL-Robotics.git
cd RL-Robotics

python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
pip install -e ".[dev]"

quad doctor
quad validate --config configs/presets/ci_smoke.toml
quad train --config configs/presets/ci_smoke.toml
quad eval --config configs/presets/ci_smoke.toml
quad bundle --run ci_smoke

One-liner (verified script):

bash scripts/quickstart-linux.sh

macOS

# Prerequisites (Homebrew)
brew install python@3.12 git

git clone https://github.com/Legendarylibr/RL-Robotics.git
cd RL-Robotics

python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
pip install -e ".[dev]"

quad doctor

# platform_linux_recommended may show FAIL on macOS — expected for sim-only work
quad validate --config configs/presets/ci_smoke.toml
quad train --config configs/presets/ci_smoke.toml
quad eval --config configs/presets/ci_smoke.toml
quad bundle --run ci_smoke

One-liner (verified script):

bash scripts/quickstart-macos.sh

Windows

# Prerequisites: Python 3.12+ and Git
# https://www.python.org/downloads/  or: winget install Python.Python.3.12

git clone https://github.com/Legendarylibr/RL-Robotics.git
cd RL-Robotics

py -3.12 -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
pip install -e ".[dev]"

quad doctor
quad validate --config configs/presets/ci_smoke.toml
quad train --config configs/presets/ci_smoke.toml
quad eval --config configs/presets/ci_smoke.toml
quad bundle --run ci_smoke

If script execution is blocked:

Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
.\scripts\quickstart-windows.ps1

Use PowerShell (not Command Prompt) for Activate.ps1. The quad command lives in .venv\Scripts\ after install. If py -3.12 fails, try py -3.11 or python -m venv .venv.

Fast smoke (all platforms)

Short run for verifying install (~30 s):

quad train -c configs/presets/ci_smoke.toml \
  -o training.total_env_steps=2000 \
  -o training.checkpoint_every_steps=1000 \
  -o task.max_episode_steps=100

Core concepts

Run identity

Every experiment is uniquely identified by:

Field	Source	Purpose
`run_name`	config	Human-readable experiment ID
`seed`	config	RNG seed for env and policy
`git_commit`	resolved at load	Exact code version
`config_hash`	SHA-256 of canonical config	Detect config drift
`trial_id`	`s{seed}` by default	Per-seed artifact suffix

Mismatch on config_hash or git_commit when resuming or comparing runs should be treated as a new experiment.

Research contract (`*_summary.json`)

Every train/eval run emits a summary containing:

learning_target — trained vs fixed components
evidence.level — tier 0–3
metric_sources — provenance tag per metric
claim_boundary — valid and invalid claims for this evidence level
reproducibility — replay command, environment capture, lockfile reference
artifact_index — stable key → path map

Validate any summary:

quad validate-summary -s runs/ci_smoke/summaries/train_ci_smoke__s0_summary.json

Privileged config keys

Safety-critical overrides require explicit acknowledgment:

quad train -c configs/presets/ci_smoke.toml \
  -o safety.torque_limit_scale=0.7 \
  --i-know-override safety.torque_limit_scale

Privileged keys: safety.torque_limit_scale, safety.estop_enabled, real.enabled, robot.urdf_path, and others — see docs/configuration.md.

Common workflows

Local development

quad list-presets
quad train -c configs/presets/dev_local.toml -o run.seed=7
quad eval -c configs/presets/dev_local.toml -n 20
quad report --run dev_local

Multi-seed paper experiment

quad train -c configs/baselines/pd_velocity.toml
quad sweep -c configs/presets/sim_baseline.toml --seeds 0,1,2,3,4 --name icra2027
quad compare --treatment icra2027 --baseline pd_baseline --metric velocity_rmse
quad methods --run sim_baseline --trial s0
quad bundle --run sim_baseline

Sim-to-real escalation

quad escalate --run sim_baseline          # gate: fall_rate < 0.05
quad train -c configs/presets/real_lab_gated.toml   # after gate passes

Audit a completed run

quad replay -c runs/<run>/config_resolved.toml
quad validate-summary -s runs/<run>/summaries/train_<run>__s0_summary.json

Configuration & presets

Configs live in TOML (human-friendly) or JSON (programmatic). Reference full config: configs/examples/full.toml.

Preset	Purpose	Evidence tier
`ci_smoke`	PR CI, no GPU, fast	1
`dev_local`	Laptop iteration, light DR	1
`sim_baseline`	Paper sim numbers, full DR	1–2
`sim_high_fidelity`	High-fidelity contact validation	2
`sweep_velocity`	Command-range ablation	1
`real_lab_gated`	Hardware smoke post-sim-gate	3
`real_field`	Outdoor tier (privileged acks)	3

Override any field from CLI:

quad train -c configs/presets/dev_local.toml \
  -o run.run_name=my_experiment \
  -o task.terrain=stairs \
  -o run.seed=42

Deep reference: docs/configuration.md · JSON Schema: schemas/config.schema.json

CLI reference

Command	Purpose
`quad doctor`	Host, asset, and dependency diagnostics
`quad validate`	Config schema + asset existence
`quad train`	Policy training loop
`quad eval`	Benchmark suites
`quad replay`	Hash-chain manifest audit
`quad sweep`	Multi-seed launch + rollup
`quad analyze`	Per-run metric aggregation
`quad compare`	Treatment vs baseline (bootstrap CI)
`quad report`	Markdown experiment report
`quad methods`	Auto-generated paper Methods section
`quad bundle`	Paper-ready export (tables, config, METHODS.md)
`quad validate-summary`	Research contract schema check
`quad escalate`	Sim→real gate check
`quad clean`	Artifact retention cleanup
`quad list-presets`	Show available presets
`quad claims`	Validate claims vs evidence tier
`quad plot`	Metric bar charts (requires `.[plot]`)
`quad sim2real`	Paired sim/real divergence check

Full flags and examples: docs/cli-reference.md

Artifacts & run output

After quad train, artifacts appear under runs/{run_name}/:

runs/go2_veltrack_v3/
├── config_resolved.toml          # frozen config + hash
├── topology.json                 # pipeline layer graph
├── logs/
│   ├── episodes/s42/ep_000001.jsonl
│   ├── manifest_s42.jsonl        # hash-chained episode index
│   └── safety_events_s42.jsonl
├── checkpoints/s42/ckpt_000500000.json
├── summaries/
│   ├── train_go2_veltrack_v3__s42_summary.json
│   └── eval_go2_veltrack_v3__s42__velocity_tracking_summary.json
├── reports/s42/report_go2_veltrack_v3.md
└── paper_bundle/s42/
    ├── bundle_manifest.json
    ├── tables/metrics.tex
    └── provenance/{REPRODUCE.md,METHODS.md}

runs/ is gitignored — copy paper_bundle/ or upload artifacts to your lab store for long-term retention.

Details: docs/artifacts.md

Evidence tiers

Tier	Label	Minimum bar	Example valid claim
0	`config_only`	Valid config on disk	"We specify PPO velocity tracking on Go2"
1	`fast_sim`	Fast physics rollouts; ≥3 seeds for comparisons	"Linear on-policy policy reduces RMSE vs PD baseline in fast sim"
2	`high_fidelity_sim`	Denser MuJoCo + slip shaping; suite MJCFs	"Measured on stairs/cobble plants (not outdoor contact validation)"
3	`real_robot`	Instrumented hardware, safety review	"RMSE < 0.2 m/s on lab treadmill"

Never claim: sim success ⇒ deployment safe; single-seed ⇒ SOTA; level-1 transfer ⇒ real-robot performance.

Full ladder and anti-patterns: docs/evidence-ladder.md

Measurement backends

Backend	ID	Tier	Status in repo
Fast MuJoCo	`mujoco_fast`	1	Real MuJoCo (required dep), coarser timestep
Contact MuJoCo	`mujoco`	2	Denser MuJoCo + slip reward shaping (not outdoor-validated contact)
Isaac	`isaac`	—	Not bundled
Real robot	`ros2_real`	3 when bound	Fails closed until ROS 2 I/O is wired

Select via config:

[sim]
backend = "mujoco_fast"   # or mujoco, isaac

[real]
enabled = false           # set true for hardware (requires acks)

Integration guide: docs/measurement-backends.md

Reproducibility & audit

Third-party reproduction requires three inputs:

git_commit from summary
runs/<run>/config_resolved.toml
Full artifact tree

git checkout <git_commit>
pip install -e .
quad validate --config runs/<run>/config_resolved.toml
quad replay --config runs/<run>/config_resolved.toml
quad validate-summary -s runs/<run>/summaries/train_<run>__s0_summary.json
quad eval --config runs/<run>/config_resolved.toml

Summaries capture reproducibility.environment (Python version, packages, platform, lockfile digest).

Protocol: docs/reproducibility.md

Development & testing

# Install with dev dependencies
pip install -e ".[dev]"

# Run test suite
pytest -q

# With coverage
pytest --cov=quad --cov-report=term-missing

# Lint config presets
quad validate -c configs/presets/ci_smoke.toml
quad validate -c configs/presets/sim_baseline.toml

# Full local quality gate (required before PR)
python3 scripts/run_ci_local.py --fast   # daily: Python pipeline
make ci                                  # test + rust-bridge
python3 scripts/run_ci_local.py --list   # recommended cross-platform matrix

See docs/CI_LOCAL.md for job steps (doctor, validate, pytest, smoke train, summary, replay).

Contributing: CONTRIBUTING.md

Troubleshooting

Symptom	Fix
`quad: command not found`	Activate venv: `source .venv/bin/activate` (Unix) or `.\.venv\Scripts\Activate.ps1` (Windows)
`PermissionError` on override	Add `--i-know-override <key>` for privileged fields
`platform_linux_recommended: FAIL`	Expected on macOS/Windows; safe for sim-only work
`git_commit: FAIL` in doctor	Install git; run inside a git clone
`ValidationError: total_env_steps`	Minimum 1000 steps; use `ci_smoke` preset or override
Windows `Activate.ps1` blocked	`Set-ExecutionPolicy RemoteSigned -Scope CurrentUser`
`py -3.12` not found (Windows)	Use `py -3.11` or `python -m venv .venv`
Real robot refuses to start	Check docs/safety.md startup checklist; set `operator_ack_token`
Resume from wrong checkpoint	`config_hash` must match; do not resume across config changes

Documentation

Full research documentation: docs/README.md

Guide	Topic
System spec	Full pipeline contract
Architecture	Layers, data flow, extension points
Configuration	Schema, presets, privileged keys
Artifacts	Output layout and manifest chain
Evidence ladder	Valid claims per tier
Reproducibility	Reviewer audit protocol
Safety	Guardrails and real-robot gates
Measurement backends	Backend I/O and binding
Paper workflow	Experiment → submission
CLI reference	All commands and flags
Local quality gate	`make ci`, `run_ci_local.py`
Baselines	PD controller + compare workflow
Metrics	Formal metric definitions
Integrations	MuJoCo, ROS 2, W&B
Roadmap	Mock vs production status

JSON Schemas: schemas/ (config, summary, topology)

Repository layout

RL-Robotics/
├── quad-bridge/            # optional Rust safety supervisor (real hardware)
├── quad/                   # pipeline implementation
│   ├── config/             # schema, loader, presets
│   ├── measurement/        # sim + real backends
│   ├── learning/           # training loop
│   ├── artifacts/          # layout, summaries, manifests
│   ├── analysis/           # aggregation, statistics
│   ├── reporting/          # reports, bundles, Methods
│   └── provenance/         # environment capture
├── configs/
│   ├── presets/            # typed experiment presets
│   └── examples/           # reference full config
├── assets/                 # URDF, scenes, skill catalog
├── schemas/                # JSON Schema for external audit
├── docs/                   # research documentation
├── scripts/                # quick-start + run_ci_local.py (local quality gate)
├── lockfiles/              # dependency pin file
├── tests/                  # pytest suite
├── Makefile                # make ci, make ci-fast, make test

Citation & license

Citation: see CITATION.cff

@software{rl_robotics2026,
  title  = {RL-Robotics: Config-First Quadruped Locomotion Research Pipeline},
  url    = {https://github.com/Legendarylibr/RL-Robotics},
  year   = {2026}
}

License: GNU GPL v3 — see LICENSE.

Navigation Menu

Repository files navigation

RL-Robotics

Table of contents

Overview

Who this is for

System requirements

Quick start

Linux (recommended)

macOS

Windows

Fast smoke (all platforms)

Core concepts

Run identity

Research contract (`*_summary.json`)

Privileged config keys

Common workflows

Local development

Multi-seed paper experiment

Sim-to-real escalation

Audit a completed run

Configuration & presets

CLI reference

Artifacts & run output

Evidence tiers

Measurement backends

Reproducibility & audit

Development & testing

Troubleshooting

Documentation

Repository layout

Citation & license

About

Releases

Packages

Used by

Contributors

Languages

Name		Name	Last commit message	Last commit date
Latest commit History 37 Commits
.devcontainer		.devcontainer
.github		.github
assets		assets
configs		configs
docs		docs
lockfiles		lockfiles
quad-bridge		quad-bridge
quad		quad
schemas		schemas
scripts		scripts
tests		tests
.env.example		.env.example
.gitignore		.gitignore
.pre-commit-config.yaml		.pre-commit-config.yaml
CHANGELOG.md		CHANGELOG.md
CITATION.cff		CITATION.cff
CONTRIBUTING.md		CONTRIBUTING.md
Dockerfile		Dockerfile
LICENSE		LICENSE
Makefile		Makefile
README.md		README.md
pyproject.toml		pyproject.toml
uv.lock		uv.lock

Folders and files

Latest commit

History

Repository files navigation

RL-Robotics

Table of contents

Overview

Who this is for

System requirements

Quick start

Linux (recommended)

macOS

Windows

Fast smoke (all platforms)

Core concepts

Run identity

Research contract (*_summary.json)

Privileged config keys

Common workflows

Local development

Multi-seed paper experiment

Sim-to-real escalation

Audit a completed run

Configuration & presets

CLI reference

Artifacts & run output

Evidence tiers

Measurement backends

Reproducibility & audit

Development & testing

Troubleshooting

Documentation

Repository layout

Citation & license

About

Resources

Contributing

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages

Research contract (`*_summary.json`)