RL Lab

A client-side multi-agent robotics lab, built as a testbed for control algorithms. Minds and bodies are configured separately and dropped into a top-down 2D arena with exact polygon collision — an empty pitch, rough terrain, or a village of buildings with rooms, doors and switches to work out. Everything is reproducible, recordable, and portable as one JSON file.

Policies are neural networks and algorithmic controllers. Both run entirely offline with no API key and no network access. Hosted language models are supported too, but they are an option, not a dependency.

Everything runs in the browser — no server, no telemetry. The same engine ships as an Electron desktop app and a headless CLI.

packages/
  core/      @rl-lab/core — physics, robots, agents, evolution, learning, schema
  web/       React + Vite app (the UI)
  desktop/   Electron shell around the built web app
  cli/       headless runner

Quick start

npm install
npm run dev            # web app on http://localhost:5173
npm run desktop        # build core + web, then launch Electron
npm run cli -- presets # environments, bodies, controllers, models

npm run cli -- test                     # 84 engine self-checks
npm run cli -- video --demo             # render a showcase of everything
npm run cli -- new --env village --name "Village patrol"
npm run cli -- new --search --env obstacle-course --name "Gait search"
npm run cli -- list
npm run cli -- run <id> --duration 60 --render

Runs and searches

There are two top-level documents, and they are deliberately not variants of one another.

run                                    one arena, simulated once
├── env             the arena
├── sim             the integrator and scoring
├── agents[]        minds   — model, weights, hyperparameters, policy, learning, memory, log
├── robots[]        bodies  — chassis, appendages, tools, sensors, eyes, channels
├── participants[]  bindings of one agent to one robot on a team
└── weights[]       weight variants available to the agents

search                                 a population search
├── base            the run every individual is evaluated in
└── evolution       population, genes, fitness

A search is not a run with a flag set: it owns a population, it breeds, and it produces generations rather than frames. Its results have a different shape, so it is a different document. Its evaluations happen to be runs.

The schema is the interface. The inspector has no tabs — it is a tree with the same shape as the document, so a parameter is always found inside the thing it configures, and a search shows the run it evaluates nested under base run.

Agents and robots are independent. The same mind can drive a hexapod in one binding and a rover in another; the same chassis can be driven by three different models in the same run. A participant is the binding: agent + robot + team + spawn point.

Older documents are migrated on load with a warning saying so — v1 (where a body and a mind were one object) splits into agents/robots/participants, and a v2 "evolutionary trial" is lifted into a search whose base is the run it was bolted onto.

Searching

Opening a search builds its first generation immediately, so the grid shows every individual — live cells for the ones inside the parallel budget, placeholders for the ones queued behind it — before you press start. That grid is how you inspect what is about to be searched; it would be useless if it only appeared once the search was already running.

A gene is a dotted path into the base run, and * addresses every element of an array:

robots.*.body.segment.appendage.motorTorque
robots.*.body.scale
agents.*.policy.decisionHz
agents.*.hparams.temperature

So anything numeric anywhere in the run is searchable without special-casing it. Selection is tournament, crossover is blend, mutation is Gaussian in each gene's own range, and elites carry over unchanged. Individual 0 of generation 0 is always the unmodified baseline, so a search can never come out worse than the configuration you authored.

A search can also evolve network weights directly, which makes it a neuroevolution run rather than a hyperparameter sweep: name the weight sets under evolve weights and each individual carries its own parameter vector, recombined by uniform crossover and mutated by a Gaussian step. Genes and weights can be searched together.

Fitness is team score, goal difference, summed agent score, distance, possession, checkpoints reached, or distance-per-joule. A new search picks one its base run can actually produce — an arena with no goals and no ball would report an identically zero team score for every individual, and the search would run to completion having learned nothing.

Learning

Each agent can learn at the end of every run, and what it learns is stored in its memory inside the saved run — so an agent genuinely accumulates across runs.

• hill-climb — a (1+1) evolution strategy. The agent perturbs its own parameters and keeps the perturbation only if the run scored better than its best so far. What it may touch is configurable: the Laplace coefficients of the robots it drives, its sampling hyperparameters, its policy loop, and — for a neural policy — the whole weight vector. Fully deterministic and offline. The authored document is never overwritten: the run uses a copy with the learned values applied, and only the memory is written back.
• reflect — a model agent is shown a summary of the run it just finished and writes one lesson, which is carried in memory and injected into the system prompt of every later run.

Both can be on at once. The memory node shows the accepted knobs, the fitness history, and every lesson; "forget" wipes it.

Policies

Every controller emits the same thing — the action vector below — so they are interchangeable and can be mixed inside one run.

network — a neural policy. The observation is encoded into a fixed-width vector derived from the robot's own configuration (its lidar rays, its tools, its control channels), pushed through an MLP with configurable widths and activation, and the output is mapped straight onto the action ranges. Optional Elman-style recurrence feeds the last hidden layer back in, giving the policy a short memory. Parameters are a flat Float32Array, so the same vector can be hill-climbed by the agent itself or evolved by a population — this is the point of the testbed.

observation → [self · ball · goals · neighbours · lidar · tools · channels · clock]
            → MLP (tanh/relu/gelu/sin/sigmoid, optional recurrence)
            → 9 Laplace coefficients per channel + 1 activation per tool

scripted — eleven algorithmic controllers (striker, goalie, defender, chaser, evader, explorer, patrol, wander, random CPG, …). Deterministic, and the baseline a learned policy has to beat.

model — a hosted language model. Kept because it is occasionally interesting, but nothing depends on it: every default ships offline.

Physics

Top-down, so there is no in-plane gravity — gravity only sets the normal load used for Coulomb ground friction.

Collision has no approximations in the narrow phase. Arbitrary outlines, including deliberately concave ones (debris piles, bushes), are ear-clip triangulated and then merged back into the fewest convex pieces (Hertel–Mehlhorn) at load time; the union is identical to the input. Collision is then exact polygon-vs-polygon SAT with reference/incident edge clipping for two-point manifolds, solved by a sequential impulse solver with warm starting.

The solver also provides revolute joints with motors and limits, a point-friction constraint (used by feet), a mouse spring (used by pointer drags), a uniform-grid broadphase, and raycasting for lidar.

Bodies

Every robot is an n-segment caterpillar, and each segment carries n legs or n wheels. A rover is 1 segment with 4 wheels. Segments are individually overridable, so a wheeled head with a legged midsection is just configuration.

Wheels and legs are real rigid bodies joined by revolute joints — they collide with scenery like anything else.

• Wheels use a friction ellipse: each wheel carries a share of the robot's weight, and that normal load buys one grip budget shared between driving and resisting sideways slide. Spending it on thrust is exactly what lets a skid-steer robot scrub its wheels round a turn. A torque–speed curve bounds top speed. Fixed wheels steer differentially; steerable wheels rotate.
• Legs grip through a solver-level ground constraint, so the traction impulse from a planted foot propagates up the hip into the chassis instead of merely cancelling the limb's own momentum. Stance is the half of the stroke cycle in which the foot sweeps backwards; duty widens that window and stance grip sets the Coulomb limit at which the foot slips.
• Limbless segments get anisotropic ground friction — low along their length, high across it. That is the snake-scale model, and without it undulation produces no net travel.

Beyond the drivetrain, the shape is yours: a hull profile (rounded, boxy, oval, wedge, hex, diamond) with a taper that narrows the nose or the tail, per-robot colours, and one scale knob for the whole machine — torque scales with it, so a bigger robot is not silently underpowered. Limbs carry a mount reach along the hull, a track width across it, a stagger between the two sides, a resting splay, and a phase shift on top of the gait pattern (offsetting one segment against the next is what turns a row of identical leg pairs into a travelling wave).

Sensing

Two feeds, and the difference between them is the point.

• The ring. sensors.rayCount rays spread over the whole field of view. Cheap, always on, sees everywhere at once — and far too coarse to resolve anything. Set it to 0 to run on eyes alone.
• Eyes. A steerable, densely-sampled cone the robot points around its surroundings. The same ray budget buys far more angular resolution than the ring, but only inside the cone, and only where the gaze happens to be.

foveation warps an even ray spread toward the cone axis: at 1, half the rays fall inside the middle sixth of the cone, so the centre of gaze resolves far finer than the periphery. saccade says what the eye does between fixations — sweep the mechanical limit, or jump to the ball, the nearest robot, the nearest threat, the opposing goal, somewhere random, or nowhere. Everything but sweep saccades: a jump at slewRate, then a fixation held for dwell.

Gaze is integrated every physics step so a saccade is a real visible sweep rather than a jump cut, but the cone behind it is only re-cast at refreshHz — a hundred-ray cone is the most expensive thing a robot does and does not need to happen at 120 Hz to be useful. Turn on eyes in the arena to see the wedge, its rays, and the fixation point; what is outside the wedge is what the robot cannot see.

Interactive scenery

Beyond rocks and crates, an arena can hold things that respond.

• Structures. A building is one object in the document and a navigable interior in the world: it expands into wall pieces with doorway gaps, recursive interior partitions, door leaves in the outer openings, furniture, and the switches that unlock them. Houses, sheds, halls, compounds, garages, corridors, towers, and braided mazes.
• Doors. Hinged, sliding, or double. A leaf is driven, not free-swinging — a kinematic body given exactly the velocity that lands it on target this step, so a closing door shoves a robot out of the doorway instead of losing to it.
• Buttons. Pressure plates: momentary, toggle, latch, or timer. They accept robots, the ball, pushable objects, or anything, optionally restricted to one team, and can be pressed from arm's length with the press tool.
• Zones. Checkpoints (paid once per robot, and orderable into a route), scoring and hazard regions paid per second, chargers, and bare triggers.

All of it is wired by signals — a named boolean the world carries. A button drives one; a door listens on one; a zone can drive one while occupied. Anything is wired to anything by typing the same name in both places, and that is the whole of the logic layer. It is enough for airlocks, keycards and pressure puzzles.

Actions

Continuous — Laplace coefficients on joint forcing functions. Each channel owns a second-order transfer function

        b0 + b1·s + b2·s²
H(s) = ─────────────────────      driven by   u(t) = sin(2π·freq·t + phase)
        1  + a1·s + a2·s²

and the joint command is bias + gain·H(s)u. The agent's outputs are those nine numbers per channel, so a policy shapes the dynamics of its own gait rather than emitting raw torques. H(s) is realised as a discrete biquad via the bilinear transform at the simulation timestep, with an explicit stability projection — an agent cannot request a divergent pole.

Channel	One per	Effect
flexN	inter-segment joint	bends segment N into N+1
driveN	segment with appendages	wheel throttle, or leg stroke amplitude/frequency
steerN	segment with appendages	steering angle, or left/right stroke bias

Appendages on a segment share their channel's coefficients but each evaluates them at its own gait phase offset, so one set of numbers drives a whole tripod.

Discrete — tool activations. Each enabled tool adds one channel in [0,1] and fires on a rising edge above its threshold, then goes on cooldown. Tools: kick, grab/release, tag, boost, scan, beacon, shield, ground marker.

A 1-segment 4-wheel rover with kick and boost has 2 × 9 + 2 = 20 action dimensions.

Models and weights

Any agent runs a built-in controller (local:striker, local:goalie, local:explorer, …) or a hosted model. The catalog covers Anthropic, OpenAI, Google, xAI, Mistral, DeepSeek, Cohere and open-weight families over any OpenAI-compatible endpoint; the model id and endpoint are free text, so anything not listed still works.

Weights are a first-class entity, separate from the model that uses them. For a neural policy a weight set is the parameter vector; for a hosted model it names a base, LoRA, checkpoint or fine-tune, and a served id is sent to the provider in place of the model id, which is how an adapter reaches an OpenAI-compatible server without any other change. Either way an agent points at a weight set, so swapping weights is a one-line change and two agents can share or diverge as you like.

Hyperparameters are per agent and filtered to what the selected model supports. Decisions are asynchronous: the robot keeps executing its current forcing functions while a request is in flight, which is what makes 4 Hz LLM control feasible against 120 Hz physics. Scripted and model-driven agents mix freely — they emit the same action structure.

Interface

Three panes: the library, the master area, the inspector. [ and ] collapse the sides. The library lists runs and searches separately, because they are different kinds of thing.

The three toolbar buttons in the middle are the three things you can be doing: new run, train, new search. Train swaps the master area for a dashboard of graphs across the whole library — search fitness by generation, run score over sim time, and per-agent learning — on three separate charts, because their units have nothing to do with each other and overlaying them on a shared scale would imply a relationship that does not exist.

Light and dark. Light is the default and the design's primary target; dark is a real second theme, not an inversion. The canvas follows it too: field colours are either authored or left to the theme (palette: auto), because a ground colour chosen to read well on black is unreadable on white.

• Click a document to open it. Click an agent to expand its parameters, its memory, or its log. Click a robot for its body, tools, sensors, eyes and channels. Nothing is behind a tab.
• Header buttons are icons. Preset pickers show diagrams, not captions — and the diagrams are generated from the real configuration (the environment glyph expands the actual scatter rules at the actual seed; the body preview is the geometry the simulation will build).
• Drag in the arena. While a run is running, dragging applies a real force through the solver. While it is a draft, dragging rearranges the configuration — scenery position, spawn point, kick-off spot. Procedural scatter is baked into explicit items the moment you drag one, because a rule cannot represent "this one, moved here".
• Wheel zooms, empty space pans, clicking a robot selects it.
• Space start/pause, ←/→ step frames (Shift for ten), T the training dashboard, Ctrl/Cmd+S save.

A document is draft → running → finished. Drafts are fully editable; running locks the parameters; finished is read-only with a scrubbable playbar and goal/kick/button/checkpoint events marked on the timeline.

Files and saving

Two shapes, both accepted on upload:

• Params only (*.params.json) — the configuration. Seeds a new draft.
• Params + results (*.result.json) — configuration plus the recording, events, per-participant statistics, per-agent logs and learning updates.

Platform	Storage
Web	localStorage, one key per document
Desktop	a directory (userData/trials, or pick your own; $RL_LAB_DIR)
CLI	a directory (~/.rl-lab/trials, $RL_LAB_DIR, or --dir)

Desktop and CLI can share one directory, so a run designed in the UI can be executed in a batch from a terminal. Recorded poses are rounded to millimetre precision and the sim node estimates the size of a run; if a long dense run will not fit in browser storage, raise record every or export to a file.

API keys

Entered under the key icon, stored only in this browser, and never written into an exported document — only the key's name is, via an agent's key reference. Requests go straight from the client to the provider, so the provider must permit browser-origin requests. Where one does not, point that agent's endpoint at a local proxy, or run from the CLI, which reads keys from the environment:

ANTHROPIC_API_KEY  OPENAI_API_KEY  GOOGLE_API_KEY  XAI_API_KEY
MISTRAL_API_KEY    DEEPSEEK_API_KEY  COHERE_API_KEY
RL_LAB_KEY_<NAME>  # a named key, matched by an agent's key reference

CLI

rl-lab test [--filter <text>] [-v]      # engine self-checks
rl-lab list                             # the library, runs and searches
rl-lab new [--env <p>] [--search]       # a draft run, or a draft search
rl-lab run <id|file>                    # simulate a run, or execute a search
rl-lab show <id|file>                   # configuration and outcome
rl-lab export <id> [--params]           # write JSON
rl-lab import <file>                    # load JSON into the library
rl-lab rm <id>
rl-lab presets

run accepts --seed, --duration, --out, --save, --quiet, --render (ASCII view of the final frame) and --no-block. It dispatches on what the document is: a run reports per-participant statistics and what each agent's learning kept or reverted; a search reports best/mean/worst per generation and the winning genome. --duration overrides the run's length, or a search's per-evaluation length.

Video

rl-lab video renders recordings to animated PNGs — no ffmpeg, no codec, no native dependency. Anything without a recording is simulated first, so "render everything I have" is one command:

rl-lab video --all                  # every saved run, plus a combined grid
rl-lab video <id> <id> --out clips  # named trials
rl-lab video --demo                 # a showcase set built from scratch

--demo builds and runs twenty-two trials — every body preset in the obstacle course, every environment, and a neuroevolution search — then writes one clip each plus combined.png, a grid of all of them advancing on one clock, and contact-sheet.png, the same grid as a still. Clips that finish early hold their last frame and dim, so it stays obvious which are still running.

The camera frames the robots rather than the pitch, scaled to how big they actually are; a robot that travels much further than fits in a readable frame gets a tracking camera instead of being zoomed out to a speck. Useful, because a 0.7 m rover on a 32 m pitch is three pixels wide.

APNG plays in any browser and most viewers. For MP4 the command prints an ffmpeg line. In the app, the record button captures the live view (arena or evolutionary grid) to WebM via MediaRecorder.

Rendering is a scanline rasterizer with 2× supersampling and a 3×5 bitmap font, about 400 lines, writing PNG through Node's built-in zlib. Frames after the first store only the bounding box of what changed, which is most of why a 48-frame 1200×782 montage lands around 2 MB.

Testing

rl-lab test runs 84 behavioural checks in a few seconds — build a world, run it, assert the thing that should be true is true — grouped so --filter physics narrows to one area:

group	what it pins down
geometry	decomposition preserves area and convexity, SAT symmetry, raycasts
laplace	discrete DC gain matches the continuous filter, stability projection holds
bodies	every preset builds, travels, and stays finite and inside the walls
physics	seed determinism, rest, joint limits, drag, surfaces, picking
controllers	all thirteen emit usable actions and none raise
tools	kick momentum, rising-edge latch, cooldown, threshold, boost
scoring	goal detection and possession
network	parameter counts, determinism, bounded output, recurrence, operators
encoding	layout matches the robot, features stay in range, round trips
paths	wildcard expansion and write-through
evolution	baseline preservation, genome write-through, fitness improves, neuroevolution
learning	best fitness is monotone, the document is not mutated, network ES
sensing	foveation density, gaze limits and refresh rate, tracking, encoding
interaction	signals open locked doors, button kinds, checkpoint ordering, buildings
serialization	round trips, v1 migration, sparse input, dangling bindings
environment	presets populate, scatter bakes and is seed-deterministic
storage	directory store CRUD, v2 → search migration, aged documents
render	rasterizer fill and clipping, PNG/APNG structure, montage, tracking
integration	full match, one mind driving two bodies, empty run

Engine bugs the suite has already caught and fixed: joints could be walked far past their stops because the limit was only solved at the velocity level (adding a positional correction halved the worst overshoot and made legged bodies walk roughly twice as far), and dense scatter rules silently delivered a third of the items asked for. Rendering caught two more by eye: a montage cell could paint over its neighbours because the rasterizer had no clip rectangle, and framing clamped to the field, so a robot that walked out of an open sandbox left an empty pitch on screen.

Determinism

A run seeds one RNG that forks per participant and per subsystem, so spawn jitter, scenery scatter, exploration noise and the genetic operators all reproduce exactly. Note that entity ids are part of that: they seed the forks that pick segment outlines and gait offsets, so the same document reproduces exactly, while re-deriving a fresh document from the same recipe does not. Scripted trials are fully deterministic. Model-driven trials are only as deterministic as the provider — set a sampling seed where one is supported, and use --no-block consistently, since whether physics waits on a decision changes the timeline.

Notes on the built-in controllers

The local:* controllers are baselines, not strong play. They exist so a run runs offline and so model agents have something to compete against. local:striker works the ball up a shooting line and scores against an empty net; in cluttered environments it spends much of its time negotiating scenery. They are the least interesting part of this codebase and the easiest to replace.

Known limits

• Joint limits are soft. They are solved before the foot anchors and the contacts, so a hard-driven gait can push through: measured across the jointed presets the worst overshoot is 0.57 rad (the centipede, whose twelve legs all torque one chain). A hard stop would need the limit solved jointly with the other constraints rather than after them.
• The local:* controllers are baselines, not strong play — see below.
• Evolutionary subtrials run in the same thread as the UI. The grid shares one frame budget across live cells, so a large population runs slower per cell rather than dropping frames.

Safety rails

Bodies are configurable down to raw torques and stiffnesses, so the solver clamps linear and angular velocity and projects unstable filter poles back inside the unit circle. A pathological configuration degrades into a slow or twitchy robot rather than one that leaves the field at 10 km/s.