platoon 0.2.0

Platoon 0.2.0 is the first release of platoon on PyPI. It replaces the original platoon: platoon now trains agents
with reinforcement learning through a Tinker-compatible training service, runs any agent harness, and serves
environments as services. The original AReaL-based trainer, its standalone inference scripts and its plugin layout
are gone, and there is no step-by-step upgrade guide.

• The original code is at the tag archive/v0.1. To install
  it from git, pin that tag: pip install 'git+https://github.com/ApGa/platoon@archive/v0.1'.
• The RAO paper's code: the experiments of
  Recursive Agent Optimization (RAO) ran on the code of the
  apga/rao-snapshot branch, which stays.
  recipes/rao carries the paper's TextCraft-Synth
  experiments over to Tinker: its single agent, its recursive agent, whose subagents are rewarded for crafting their
  own targets (the paper's Dense rewards), and its Sparse ablation, in which every subagent takes the root's success.
• Why 0.2.0: the original platoon gave its version as 0.1.0, so the two never share one.

Install

Python 3.12 or later (CI tests 3.12, 3.13 and 3.14), on Linux, in a new virtual environment:

pip install 'platoon[tinker,ors]>=0.2,<0.3'
uv add 'platoon[tinker,ors]>=0.2,<0.3'      # in a uv project

Add hpc, as in platoon[tinker,ors,hpc], for Singularity/Apptainer or enroot sandboxes on clusters.

Extra	Installs	Needed for
tinker	the Tinker SDK (0.16.1) and tinker-cookbook (0.3)	training and evaluation
ors	the openreward SDK	ORS environments, including the examples
hpc	hpc-sandbox (0.3), from PyPI	Singularity/Apptainer and enroot task sandboxes, for Harbor datasets on clusters

• Keep the range when you pin platoon. Below 1.0, a minor release (0.3.0) may change the Python API, the config
  format or the command line, and a patch release (0.2.1) only fixes bugs. The range also makes pip replace an
  original platoon 0.1.0 that is already installed.
• A project that trains with platoon names it as a dependency with the same range:
  Depend on platoon from a project.
• torch comes from PyPI as its CUDA build, with several GB of CUDA libraries that platoon, which runs on the CPU,
  does not use. uv takes the CPU build instead: uv pip install --torch-backend cpu 'platoon[tinker,ors]>=0.2,<0.3'.
• TextCraft is the first-party extension platoon-textcraft 0.1.0, on PyPI from its own tag:
  pip install 'platoon-textcraft[platoon]'. It also exports TextCraft-Synth as a Harbor dataset.
• The tutorials, examples and recipes are in the repository, not in the package. Clone it at the tag:
  git clone https://github.com/ApGa/platoon.git && cd platoon && git checkout v0.2.0 && uv sync --extra tinker --extra ors.
  Number search, the first run's environment, ships in the package as the built-in named environment
  platoon-number-search@1.

What it does

Training

• Reinforcement learning through any Tinker-compatible training service, hosted Tinker or a self-hosted server
  such as miles-tinker. platoon train runs groups of episodes per task,
  builds training data from the exact sampled tokens and centers rewards within each group (mean or leave-one-out
  baselines, with depth weighting for recursive agents).
• Every training task trains. Tasks are drawn in seeded shuffled passes that go on from one epoch to the next, so
  no task is left out of every epoch. Several environments entries make a mixture, whose tasks training interleaves
  in a weighted fair order: weight: tasks, the default, draws every task of the mixture equally often.
  platoon train logs each entry's share, and platoon config validate --tasks prints the same before a run.
• Metrics by label. train.label_by names the dataset column that labels the tasks, by default the mixture
  entry. Each step's metrics and each evaluation's break down by its labels, and group limits and straggler
  policies key on them.
• Evaluation that rolls out as training does. A training config's eval.workflow_config is laid over
  train.workflow_config: an evaluation during training keeps training's rollout and rollout-worker settings unless
  it sets them.
• Cadences in training steps, concurrency in rollouts. eval.every and checkpoint.every count training steps,
  and 0, the default of both, does neither. The first evaluation is at step every; eval.at_start: true also
  evaluates the starting policy, at step 0, and a run that resumes does not evaluate a step twice.
  max_concurrent_rollouts counts the rollouts that run at once. A training config without an eval section does
  not evaluate.
• One run per run directory. A run records its model, renderer and held-out tasks (identity.json), and a
  resume that would change them stops before it opens the training service.
• Runs that start again by themselves. With watchdog.max_restarts: N, platoon train starts again from the
  newest checkpoint after a failure, and reaps the sandboxes its run left. platoon sandbox reap RUN_DIR reaps those
  of a run that was killed.
• A missing or refused Tinker key is one line. platoon train, eval and rollout check TINKER_API_KEY
  before they prepare, write or reap anything, and a key that is missing, malformed or refused by the service ends
  them with one line and exit 78 instead of a traceback.
• Recovery checkpoints that do not fill the disk. With checkpoint.keep_recovery: N, training deletes its
  recovery checkpoints beyond the newest N: never a permanent one, the one a resume starts from, or one that
  rollouts may still sample from.
• Outcomes that say nothing about the policy (a crashed grader, a dead sandbox, a timeout) never train as a
  reward of 0.
• Subagents. An agent starts subagents with ctx.agents.spawn, under step budgets and depth limits. One rollout
  trains every agent in the tree, and episode programs (@pt.program) compose episodes and shape rewards.

Agents, models and harnesses

• Any harness: a Python function (@pt.harness) or a command-line agent described by a TOML spec, whose
  options a config sets for each environment (harness_options). Built in: platoon-codeact, platoon-react,
  platoon-bash, mini-swe-agent and openhands, and platoon-oracle and platoon-noop, which run a task's
  reference solution or nothing, to check a dataset before training.
• Harnesses inside task sandboxes, also sandboxes without a network, from toolchains built once per site.
• A recording model gateway. Every model call, from Python (ctx.chat) or from any client over
  OpenAI-compatible HTTP, is rendered, sampled, parsed and recorded with its exact tokens, and an MCP tool plane
  serves each agent the environment's tools. The OpenAI Responses and Anthropic Messages dialects are experimental.
• The sampling timeout in the config, beside the model: train.sample_timeout_seconds and
  inference.sample_timeout_seconds. Without them, TINKER_SAMPLE_TIMEOUT_SECONDS sets it, else 600 s.
• Tool calls for any chat template. The chat_template renderer renders with the model's own template and reads
  tool calls with a parser chosen by name, found by a round trip through the template (tool_call_parser: auto), or
  written by you.
• Evaluation with hosted models. platoon eval and platoon rollout evaluate a model behind any server that
  speaks the harness's own dialect (inference.hosted), and the key never reaches the agent's sandbox.
• A directory for each evaluation. platoon eval writes each start to a new directory,
  runs/eval/<policy>/<UTC time> (under the site profile's runs when it sets one), unless inference.output_dir
  names one, and never resumes another policy's rollouts.

Environments and sandboxes

• Environments as services: OpenReward (ORS) environment classes, in process or on environment servers; Harbor
  task directories, graded by their own tests in process, Docker, Singularity/Apptainer or enroot sandboxes; step
  environments through pt.gym; and task tables.
• Harbor datasets as published. A Harbor environment names a dataset (hf://org/name@commit, or a local
  directory) that platoon harbor fetch downloads once and pins with a lock, and platoon harbor images builds its
  tasks' images once per site, also from Dockerfiles that ADD a URL or a git repository, which the oci builder
  builds with BuildKit as Docker does. holdout holds out whole groups of tasks for evaluation, so that no held-out
  task holds data that training has seen. The README's
  Train on a Harbor dataset goes
  from a site profile to training with configuration alone, and
  examples/harbor-dataset is that project,
  ready to copy.
• One command prepares a config. platoon prepare CONFIG makes, once per site, what the config's runs need:
  datasets, images, toolchains and the tokenizer. A run whose toolchains or images are missing stops before any
  rollout and names the command.
• Harbor grading that agents cannot forge. Grading kills every process the agents left running, makes the tests'
  and the reward's directories afresh, and reads only regular reward files.
• Rollout deadlines from the task. Without a rollout_config.timeout, a rollout's deadline follows from the
  harness's own time limit and a Harbor task's [agent] timeout_sec, so a Harbor config needs no timeout.
• Rollout workers that start in seconds, forked from a fork server that imported their stack once
  (worker_start_method: forkserver, the default), also on GPU nodes. With hpc-sandbox 0.3, the singularity
  provider's rootfs_layout: fuse-overlay starts a sandbox in about a second.
• Nothing a stopped rollout started keeps running. Every sandbox carries its run's label, so the sandboxes that
  a killed worker or run left are reaped by label, and no other run's.

Configuration and the command line

• One command line: platoon train, eval, rollout, prepare, config, harness, program, env,
  extensions, harbor, sandbox, toolchain and viz. Configs are strict and build on each other through
  extends:, a run refuses (exit 78) a config whose rollouts could never train, and platoon config validate checks
  a config as a run would.
• A site profile. ~/.config/platoon/site.yaml, or the file PLATOON_SITE names, says where a site keeps
  platoon's files and which runtime starts its sandboxes, so a project's configs name no site path.
• Paths relative to the file that names them. A config's input paths resolve against the config file, as
  extends: does, so a config runs the same from any directory.
• A lock.json in every run directory records the versions of platoon and its main dependencies, the options
  each harness ran with and the components the run named.
• Deprecations that say what replaces them: a pt.PlatoonDeprecationWarning names the key, its replacement and
  the release that removes it, and platoon config validate --strict fails on them.

Extending platoon

• Extensions. Installed packages provide harnesses, episode programs, named environments and reward processors
  by name through entry points, which platoon imports only when a config names them. platoon extensions list,
  show and check list them, describe one and check them.
• platoon.testing runs environments, harnesses and programs on the CPU with a scripted policy and a toy
  tokenizer, with no model download.
• The package ships type information (py.typed).

Deprecated, and removed in 0.3.0

Each still works in 0.2.0, with a warning that names what replaces it.

• rollout_config.propogate_root_success: use propagate_root_success.
• A fetch config file for platoon harbor: say the same in an entry of the run config.
• python -m platoon.train.tinker.restart_wrapper: use watchdog.max_restarts.
• train.rollout_group_key: use train.label_by. workflow_config.straggler_policy_key: straggler policies key on
  label_by's labels.
• The strategy of eval and checkpoint: every alone counts training steps. The
  num_concurrent_rollout_workflow_workers of train and eval: use max_concurrent_rollouts, the old value times
  the section's workflow_config.group_size.
• A Harbor entry's cache, image_map and toolchain_root, and the variables PLATOON_HARBOR_CACHE and
  PLATOON_TOOLCHAINS: use the site profile's cache.

Documentation

• The documentation: getting started, tutorials, how-to
  guides, extending platoon, a command-line, config and Python API reference, and the RFCs.
• CHANGELOG.md: the full notes for 0.2.0.
• CONTRIBUTING.md: development, the checks CI runs,
  and how a release is cut.