Skip to content

Releases: ApGa/platoon

platoon 0.5.0

Choose a tag to compare

@ApGa ApGa released this 29 Oct 04:12
efe3b27

Harbor's multi-container tasks run: a task whose compose file names services besides main (a database, a judge, an MCP
server) runs as a stack, on Docker as a docker compose project, on Singularity, Apptainer and enroot as a rootless pod
(experimental), on the host network where pods cannot run, and on the process provider, and a fetch now locks such
tasks as runnable. Harbor datasets come from the Harbor Hub, git repositories and upstream adapters; a task may run as
the users it names (agent_user: honor), on Docker and on Singularity with fakeroot; and node services serve every
rollout on a node, as the judge of the new FrontierSmith recipe does. Training gets an extensible trainer API
(experimental): one lifecycle for every trainer, step hooks, supervised fine-tuning (trainer: sft), credit estimators
per environment entry with on-policy distillation from a frozen teacher, per-datum stale masks, several policies in one
run (frozen, trained in lockstep, or each stepping on its own clock), and a tool allowlist per agent role. The training
code moved out of platoon.train.tinker into packages named after the stages of a run; the old module paths keep
working, with a deprecation warning, until 0.6.0. The hpc extra requires hpc-sandbox 0.5.1. The TextCraft extension's
release for platoon 0.5 is platoon-textcraft 0.4.0.

New features

  • Harbor datasets from the Harbor Hub, git repositories and adapters. A Harbor environment's dataset may name a
    package of the Harbor Hub (harbor://org/name@ref, a dataset or a single task, at a revision, a sha256: digest
    or a tag), a dataset of Harbor's legacy registry (harbor://name@version, from Harbor's registry.json, or the
    file or URL registry names), or a directory of a git repository (git+<url>@<commit>#<path>), and adapter: {repo, commit, path, command, args, env, sparse} in its place generates the tasks with an upstream adapter at a
    pinned commit, as Frontier-CS's and Harbor's own adapters do. The Hub's packages come through the Harbor CLI at
    platoon's pin (harbor==0.24.0, which platoon runs with uvx and never imports; PLATOON_HARBOR_CLI names
    another), resolved to their digest and downloaded by it, and each task's files must hash to the digest the Hub gives
    it; a registry's tasks and a git directory come through shallow, blobless fetches that check out only the selected
    tasks, and an adapter runs on a checkout of its repository at the commit (whole, or as sparse says), its output
    fetched as a local directory. Everything else is as for any fetched dataset: selection, overlays, Harbor's content
    hash, the stack plans of compose files and the stacks they run as (below), the lock, which now also records how the
    source resolved (source), and rollout workers that read only the lock and the cache. A Hub revision or digest, a
    full git commit, an adapter's commit and a version in a registry file are pinned, so that platoon train and
    platoon eval fetch them themselves; a tag, a branch, a registry task at HEAD and Harbor's own registry move,
    and platoon harbor fetch CONFIG --update resolves them into the lock (Fetch a dataset by
    name
    ). A symbolic link that leaves its task fails the
    fetch, so that no file of the fetching host reaches the cache, and a URL with credentials is refused; registry and
    a local adapter.repo are relative to the config file, as every input path is.
  • Harbor compose files are planned when a dataset is fetched, and those that name only main run. A task's
    environment/docker-compose.yaml is read as Harbor layers it, with compose's interpolation and Harbor's variables in
    platoon's meaning (CONTEXT_DIR, the log directories' HOST_*_PATH, CPUS, MEMORY), into a stack plan that the
    lock records with each task (compose, under the lock's compose_rule), and its kind in flags.stack: single,
    trivial, the new main_only, or sidecars. A file that names only main and gives it settings runs as the task's
    one sandbox on every provider: its environment, working directory, resources and binds (the task's own files,
    read-only from the cache, a writable bind as a copy; volumes below a bind keep what they mask out of the sandbox), its
    healthcheck, and its command, started as main's init, which the kill before grading spares with what it started;
    an init that is gone by grading makes the attempt an error (main_exited), and a copy fork of such a task is
    refused. A file outside the subset platoon runs, security-relevant keys such as privileged or a cap_add of
    NET_ADMIN included, is refused at fetch, naming the service and key (compose: <service>.<key>: ...). A dataset's
    overlay may edit compose services ([[compose]]: set, replace or unset their keys, or mark one external) and list
    the host variables a compose file may name (compose_host_env), which sessions resolve when they open, so that no
    host value enters a lock; a read-only bind may name the path a listed variable names, and no other host path. A
    file with services besides main runs as a stack (below)
    (Compose files).
  • Sandboxes can keep a task's processes through kill_processes. Sandbox.keep_processes(entries), optional, spares
    processes a session started for the task before any agent ran, with their descendants; the built-in providers have
    it (Write a sandbox provider).
  • Harbor tasks that are multi-container stacks run. A task whose compose file names services besides main (a
    database, a judge, an MCP server) runs as a stack on a sandbox provider with the new stacks capability: main is
    the sandbox a main-only file would give the task, and each sidecar a sandbox of its own, with its image, command
    (compose's over its image's), environment, healthcheck, volumes (named ones shared
    by the services that name them), binds (writable ones as one copy of the task's files that every service binding it
    shares, which the stack owns), ports and names. The services start in dependency order, under compose's three
    conditions (service_started, service_healthy, and service_completed_successfully for init jobs, each bounded
    by sandbox_start_budget_s), and the session opens once all of them are up, as docker compose up --wait does, and
    a last probe finds them so; a sidecar that fails, exits or never turns healthy fails the session's start
    (sidecar_start, an error), and nothing of the stack is left, also when the start is cancelled. Grading runs the
    task's [[verifier.collect]] hooks in their services and probes every sidecar before and after the tests (before
    and after the artifacts are collected, for a separate verifier, which also receives the artifacts that
    artifacts[].service names from a sidecar's own files): a sidecar that is down makes the outcome invalid
    (sidecar_unhealthy, with its name, state and log), whatever the reward files say, unless the environment sets
    on_sidecar_down: grade. What a sidecar writes in a log directory it binds (${HOST_VERIFIER_LOGS_PATH}) reaches the
    tests after grading made that directory afresh, so the agents can neither erase nor forge it. shared forks share the
    stack and copy forks are refused; a rollout's deadlines count the services' starts and health waits (their images'
    healthchecks too) along their dependencies' longest path; outcomes record each service's run mode and the state
    that the grade's probes found (services). On the process provider, which isolates nothing, each declared port is
    a port outside the kernel's ephemeral range that the stack holds for its life, which a service learns from
    PLATOON_PORT_<port>, and a reference to a service at a port it declares (scheme://<service>:<port>,
    <service>:<port>) in environment values, commands, healthchecks, collect hooks, [verifier].env and the task's MCP
    servers' URLs reaches it, as does a service's own localhost:<port>. stacks: refuse refuses every stack, and so
    does a provider without the capability (singularity and enroot where pods cannot run, below); a fetch locks stacks
    as runnable unless its refuse sets sidecars: true (Changes).
    Stacks that no rollout could start (a dataset directory's registry sidecar that its compose file gives no command, or
    that builds from a context, neither of which a directory's missing image map can give) are skipped by a dataset
    directory and stop a run when a fetched dataset's tasks are listed
    (Multi-container tasks).
  • A Harbor environment can run every sandbox without a network. network_policy: none gives every sandbox and
    stack of the environment no network, in every phase, for datasets whose answers the web holds; task, the default,
    gives each phase the network its task asks for, and a stack graded in its own main gets the verifier's
    (Multi-container tasks).
  • Write a provider that runs stacks. platoon.sandbox.stacks defines the stack spec (StackSpec, ServiceSpec,
    Healthcheck, Dependency, Volume), the running Stack (main, services, health(), logs(),
    set_network(), connect(), rewrite(), stop()), StackProvider.start_stack() and the stacks capability, and a
    generic driver, drive_stack, that starts a stack's services in dependency order over a backend that starts one at a
    time, rolls back on a failure or a cancellation, and probes them
    ([Write a sandbox provider](h...
Read more

platoon-textcraft 0.4.0

Choose a tag to compare

@ApGa ApGa released this 29 Oct 04:39
efe3b27

platoon-textcraft 0.4.0 is the TextCraft extension's release for
platoon 0.5: its platoon extra requires
platoon[ors]>=0.5,<0.6, where 0.3.0's requires platoon 0.4. Its code and data are those of 0.3.0. The release for
platoon 0.4 stays 0.3.0: there, keep the extension within 0.3 (platoon-textcraft>=0.3,<0.4), since platoon 0.4
refuses the components of 0.4.0, also when it is installed without its extra. It is a minor release, not a patch
release, because it moves the platoon it installs to a new minor release.

Install: pip install -U 'platoon[tinker,ors]>=0.5,<0.6' 'platoon-textcraft[platoon]' (the environments alone, for any
ORS client: pip install platoon-textcraft).
Full notes: CHANGELOG.md and the
extension's README.

platoon 0.4.0

Choose a tag to compare

@ApGa ApGa released this 25 Oct 07:41
b46d0fc

Training can choose its tasks by their pass rates and train only on the groups that carry signal: task samplers per
environment, an adaptive pass_rate sampler, an API for task samplers of your own, and group admission with refill, all
experimental. Rollout permits now come back as each rollout ends, so the next group starts on the permits that a group's
finished rollouts give back: that is the new default, train.permit_release: rollout (group trains as 0.3 did), and
in process a permit also waits for its rollout's background teardowns. Rollouts start sooner: platoon train stages
harness toolchains and preloads the rollout workers' modules before the first rollout, and a limit on rollouts starting
at once is new and experimental. train.task_order, which 0.3.0 renamed train.task_sampler, is removed. A group now
cancels its other rollouts when one of them raises, and the warning about a model service that sends no routing ids
comes once, not for each rollout. The TextCraft extension's release for platoon 0.4 is platoon-textcraft 0.3.0.

New features

  • Task samplers you choose per environment, and an adaptive one (experimental). Each environment entry may set
    task_sampler and task_sampler_options: under train.task_sampler: weighted (a mixture's default), the run hands
    each slot to an entry by weight, as before, and the entry's sampler chooses which of its tasks fills it. uniform,
    the default, draws the entry's tasks in the seeded passes it always has; pass_rate chooses the task whose next
    group is most likely to mix passes and failures, so that more groups train, from the per-task statistics: Thompson
    sampling on each task's pass rate, with a fifth of the draws going to the tasks drawn least recently and at most
    three draws of a task per pool-sized window. train.task_sampler: pass_rate chooses over every task of the run
    instead. Each step logs what it drew and why, the informative share it predicted and the one it got, how
    concentrated its draws are, and how long it took to decide (train/sampler/...). It is experimental: it becomes the
    recommendation only after a GPU comparison with uniform shows more informative groups at an unchanged held-out
    pass@1
    (Choose tasks by their pass rates).
  • Write your own task sampler (experimental). A sampler is a small class, registered with
    @task_sampler(name, options=...) from an entry's package, or named as module:Class, at the run level or an
    entry's, with typed options (train.task_sampler_options, or the entry's task_sampler_options) checked against its
    schema before the run starts (core's samplers' already when the config loads) and recorded in lock.json. A sampler
    that reads no statistics draws when the order reaches it, as the built-ins always have; one that reads them decides
    when its group can start, among the tasks whose label has room, reproducibly from its seed and its decision count.
    The API and the per-task statistics types it reads are in the new platoon.experimental module, and
    platoon.testing.check_task_sampler checks a sampler against the contract
    (Write a task sampler).
  • Train only on the groups that carry signal, with refill (experimental). train.group_admission: informative (or
    an environment's own group_admission) decides when each training group settles whether it counts toward the batch: a
    group whose workflow says it trains nothing (its rollouts scored alike, or it carried no datums) is rejected, and one
    whose rollouts scored alike frees its place at once, so that another group starts while the trainer works (one without
    datums, which can end at once, keeps its place until a step takes it). A step trains the first batch_size admitted
    groups that are not stale, and stops waiting once it has counted refill.max_groups_per_step (3.0) times batch_size
    groups, training what it admitted. group_admission_options: {band: [lo, hi]} also rejects groups whose pass rate is
    outside the band. all, the default, admits every group, as before. A step then trains up to several times the
    informative groups, and only those, at more rollouts a step: up to the cap's multiple of the groups it counts, and
    where the trainer bounds the run, the groups that start while it trains, which the horizon and the rollout permits
    bound, not the cap; each step logs what it admitted and rejected, and why (train/admission/...). Under a loss whose
    objective is not zero at zero advantages, a rejected group's advantage-independent terms go too, and loading warns;
    beside a rule, num_minibatches must be 1. Custom rules register with @group_admission or are named as
    module:Class; the API is in platoon.experimental, and platoon.testing.check_group_admission checks a rule. It is
    experimental: it becomes the recommendation only after a GPU comparison (Train only on groups that carry
    signal
    , Write a group admission
    rule
    ).
  • Rollout permits that come back as each rollout ends, by default. A training rollout returns its permit
    (train.max_concurrent_rollouts) as soon as it ends rather than when its group's workflow returns
    (train.permit_release: rollout, the default), so the next group starts on the permits a group's finished rollouts
    give back instead of waiting for its slowest one; on long agentic tasks a quarter to a third of the budget sat idle
    that way. GroupRolloutWorkflow runs each member on its group's lease, so a group starts on its first free permit
    and its members as the rest free (rolling starts), each member's straggler deadline shifted by its own start delay; a
    workflow that does not say it runs its rollouts on the lease gets each group's permits whole, as before. Groups are
    drawn, decided, credited, admitted and resumed as before, a partly finished group counts whole toward the horizon,
    and a run at max_staleness: 0 with the default budget trains the same datums in the same order. More groups are
    open at once, so more long groups can go stale: the run warns when they do, and raising max_staleness by one is the
    documented answer. The default changes ahead of a GPU comparison of the two settings; train.permit_release: group
    keeps the behavior of earlier releases, event for event (RFC 0020;
    Rollout permits; what changes for a config that sets nothing is
    under Changes, below).
  • Run a workflow's own rollouts on the lease (experimental). platoon.experimental exports rollout_lease() and
    RolloutLease: a workflow runs each rollout in async with rollout_lease().rollout(i):, whose permit comes back
    when the block exits, and declares runs_rollouts_on_lease = True for rolling starts. In a training draw a block for
    an index outside the draw's size raises IndexError, and a second block for an index RuntimeError, under either
    setting; outside a training draw, as in an evaluation, the lease is a no-op (Run rollouts on the
    lease
    ).
  • See what keeps groups from starting. Every training run logs, each step, the share of the step in which the
    dispatcher could not start a group and why (train/dispatch/blocked_share/{permits,horizon,labels,plan,sampler}),
    the permits held (rollout_client/permits_held_mean), the groups waiting for a worker-startup slot under process
    isolation (rollout_client/startup_gate_waiting_mean), and, where a straggler policy applies, what it censors
    (train/batch/censoring/*, and each label's: the share of rollouts it cancelled, and over mixed groups the pass
    rate of each group's slowest valid rollout against the others'). It warns, once per start of training, when more
    than 2% of the groups the last 10 steps consumed went stale (so from the tenth step on), naming what held the
    dispatcher back and the fixes, and when more of GroupRolloutWorkflow's rollouts ran at once than
    max_concurrent_rollouts.
  • Rollouts start without the work that each used to do again, or wait for. Two costs fell on a rollout's start.
    A CLI harness's first rollouts on a node waited while one of them copied the harness's toolchain to node-local disk
    (about 36,000 files and 800 MB for OpenHands), at every start and every resume in a new job: platoon train now
    stages its toolchains while it opens the training service and saves at launch, and starts no rollout, training's
    or an evaluation's, before that is done (platoon eval while it lists its tasks), so that no rollout waits for the
    copy and the first ones wait only for what of it outlasts the launch save; it waits ten minutes at most, and not
    past a shutdown request. And each rollout worker imported, after its fork, what its environment and harness import
    when they open, about 3,700 modules for an in-process ORS environment played by CodeAct, and parsed the
    environment's task file again: the workers' fork server now does that once, before it forks them, from what the
    config names (the environment's module and the openreward SDK, each function harness's and program's module, the
    registry lookups, an in-process environment's task lists for the run's splits), and starts as soon as the training
    client is open. It keeps only what leaves it fork-safe: a module or a build that starts a thread or a process,
    leaves a file, socket or event loop open, or seeds a numpy generator from the OS is left out with a warning t...
Read more

platoon-textcraft 0.3.0

Choose a tag to compare

@ApGa ApGa released this 25 Oct 08:05
b46d0fc

platoon-textcraft 0.3.0 is the TextCraft extension's release for
platoon 0.4: its platoon extra requires
platoon[ors]>=0.4,<0.5, where 0.2.0's requires platoon 0.3. Its code and data are those of 0.2.0. The release for
platoon 0.3 stays 0.2.0: there, keep the extension within 0.2 (platoon-textcraft>=0.2,<0.3), since platoon 0.3
refuses the components of 0.3.0, also when it is installed without its extra. It is a minor release, not a patch
release, because it moves the platoon it installs to a new minor release.

Install: pip install -U 'platoon[tinker,ors]>=0.4,<0.5' 'platoon-textcraft[platoon]' (the environments alone, for any
ORS client: pip install platoon-textcraft).
Full notes: CHANGELOG.md and the
extension's README.

platoon 0.3.0

Choose a tag to compare

@ApGa ApGa released this 20 Oct 05:38
75685e8

Each environment of a training mixture can set the training settings of its own groups, training keeps per-task
statistics and logs pass@k, a dispatcher starts each training group when it can start, and a resume takes up exactly
where the run stood. A label at its rollout_group_limits no longer paces the run, which changes what such a config
trains. Harbor verifiers that ask for root can run as root (verifier_user: honor), and the openhands harness can
condense at a token count. train.task_order is now train.task_sampler, with a new uniform sampler. The forms that
platoon 0.2.0 deprecated are gone, but strategy stays, as the unit of every. The hpc extra takes hpc-sandbox 0.4,
and the TextCraft extension's release for platoon 0.3 is platoon-textcraft 0.2.0.

New features

  • Each environment of a mixture sets the training settings of its own groups. An entry's workflow_config is laid
    over train.workflow_config for its groups: its group_size, its baseline (leave_one_out_baseline), its
    min_successful_group_size, its straggler defaults, its filters (filter_errors, filter_message_actions) and
    subagent datum sampling, and its rollout settings (rollout_config: max_steps, timeout, sampling and the rest),
    so a dataset's calibrated step budget and a slow environment's fewer, longer rollouts sit beside the run's. Its
    rollout settings are its evaluations' defaults, below eval.workflow_config (and in platoon eval, below
    inference.workflow), and eval_overrides.workflow_config sets what its evaluation rollouts change (every split but
    train). An entry's max_staleness raises the staleness bound of its own groups, while train.max_staleness stays
    the bound on how far rollouts run ahead, and recovery pruning keeps the checkpoints within the largest bound. Loading
    refuses a key that acts on microbatches or on the run's rollout workers with where to set it, an entry bound below the
    run's, and an entry's settings beside a workflow class that does not apply them (exit 78). Named environments may ship
    rollout settings (workflow_config.rollout_config) only. platoon config show prints each entry's effective settings
    with the layer each came from (--entry NAME for one), platoon config validate and platoon train say what each
    entry changes, and lock.json records it (entry_settings). max_concurrent_rollouts counts rollouts that groups of
    different sizes share, and each step logs each label's stale_groups and num_steps
    (Settings per environment).
  • Training keeps per-task statistics. Each settled group of a training task is recorded with the workflow's
    verdict on what it trains (trained, uniform, or why it trains nothing) and what each rollout returned, passes
    judged as platoon eval judges success: on the reward the environment scored, before any reward processor, against
    the entry's new success_threshold (1.0 by default; a named environment may ship its own), which only these
    statistics read (platoon eval keeps inference.workflow.success_threshold). Each step logs each label's mean
    expected pass@1 and pass@G under decayed per-task counts (train/tasks/<column>/<label>/pass_at_1, pass_at_G), the
    share of its tasks seen, its tasks by pass-rate band and those without evidence, and the kinds of the groups it
    trained on (train/batch/informative_groups, all_pass_groups, all_fail_groups, uniform_mixed_groups, also by
    label). Each full checkpoint saves the statistics to tasks/ in the run directory and names the file in its
    checkpoints.jsonl row, a resume loads them before its launch save (from a checkpoint without them, it goes on from
    the newest older one that has them, else from empty, and lock.json lists the start under tasks_state_lost),
    recovery pruning deletes a pruned checkpoint's file, and tasks/events.jsonl logs every draw, settle and fate. The
    statistics never fail training. platoon tasks show and platoon tasks export read them
    (Per-task statistics).
  • A resume takes up exactly where the run stood. Each full checkpoint saves where the run's draws stand in a file
    of its own (tasks/dispatch-s<start>-<checkpoint>.json, which its row names as dispatch), and the groups that
    settled and that no step had trained yet (tasks/kept-s<start>-<checkpoint>.json.gz): a resume trains those as they
    are, a failed one as failed, without rolling them out again, draws again first the tasks whose groups were running
    or waiting, and goes on with the order where it stopped. So every position of the order is consumed exactly once,
    trained, dropped as stale, failed or skipped, and an interrupt or a crash no longer reruns hours of a slow
    environment's groups or loses the groups in flight, also when the per-task statistics stopped or could not be
    written. Every full save writes the groups it keeps again, about ten bytes a token, in a thread beside training;
    checkpoint.keep_recovery deletes them with their checkpoint. A resume whose sampling inputs changed (the task
    sampler, the batch size, the label column, the entries' order, an entry's weight, seed or tasks) warns, records it
    in lock.json (sampling_changed) and goes on; one from a checkpoint of an earlier platoon draws from its batch,
    as before (Checkpoints and resume).
  • train.task_sampler: uniform (train.task_sampler is the new name of train.task_order, below) draws every
    training task of every entry in seeded shuffled passes, with no entry slots: every task equally often, as
    weight: tasks does, with batches as the shuffle mixes them rather than balanced by entry. It reads no entry's weight
    or seed: a config that gives an entry a numeric weight does not load (exit 78). For a single environment it draws the
    batches auto draws, in epochs that hold every task
    (Train on several environments).
  • Harbor verifiers that run as root. Some Harbor tasks grade as root ([verifier] user = "root") and run the
    agent's code as another user, nobody as a rule, so that it cannot read the files the grader keeps for itself, such
    as held-out answers. In a rootless sandbox their tests run as your own uid, which cannot switch users: they score
    every attempt 0, the reference solution's included, or run the agent's code where it can read those files.
    verifier_user: honor now runs such a task's separate verifier as root, in a sandbox started with
    verifier_provider_options laid over provider_options: on Singularity, {rootfs_layout: copy, fakeroot: true},
    whose --fakeroot maps root to your uid and your subordinate ids to the other users, so nothing gains a privilege
    on the host. On Singularity it needs hpc-sandbox 0.4, whose provider takes fakeroot and gives back what the other
    users write before it deletes a sandbox, and which the hpc extra now installs (below); with an older hpc-sandbox,
    such as 0.3.0, such a run is refused.
    verifier_user (refuse, ignore or honor) decides for [verifier].user when it is set, and without it
    agent_user decides for both users, as before. With a fetched dataset, set refuse: {agent_user: false} too: the
    fetch otherwise leaves out every task that names a user, and a run with honor warns of the root verifiers it left
    out. A run refuses, when it lists its tasks (exit 78), a root verifier whose sandboxes cannot give it root and the
    other users (singularity without fakeroot, or with an hpc-sandbox that does not take it, enroot, which maps only
    your uid, and process), and, for a fetched dataset, a [verifier].user that honor cannot honor (another user
    than root, or a root verifier of a task graded in its agents' own sandbox), which a dataset directory skips with a
    warning. Every started root verifier's sandbox is checked to run as root with every user up to nobody mapped, and
    to let nobody run the image's shell, else the grade is an error (verifier_start). A root verifier grades as
    Docker would: with umask 022, Docker's modes for the directories platoon makes and for a working directory the
    image lacks, the image's own modes for those it has, its artifacts readable by every user, and the outcome records
    verifier_user: root. Separate verifiers' sandboxes are reaped with their own options too
    (Verifiers that run as root).
  • The openhands harness condenses at a token count. Its new option condenser_max_tokens gives its summarizing
    condenser a token trigger, and runs that condenser whatever condenser says: before each model call, it condenses
    once the prompt the agent would send is longer than that many tokens, summarizing the oldest turns after the system
    prompt and the task until the rest fits in half the trigger. condenser_max_tokens: 104857 condenses at 80% of a
    131072-token context window; a trigger that a prompt passes with nothing left to forget, such as one below the
    agent's first prompt, can never be met, and is set aside with a line in the harness log. The runner asks the
    gateway for each count (POST /v1/messages/count_tokens), so it is the length of the prompt the gateway renders
    with the policy's renderer; where a hosted model's provider gives no count, the SDK's estimate counts, and a prompt
    that still overflows the window condenses as before. The summaries are the agent's aux calls (below), trained
    with the rollout's advantage. The d...
Read more

platoon-textcraft 0.2.0

Choose a tag to compare

@ApGa ApGa released this 20 Oct 05:38
75685e8

platoon-textcraft 0.2.0 is the TextCraft extension's release for
platoon 0.3: its platoon extra requires
platoon[ors]>=0.3,<0.4, where 0.1.0's requires platoon 0.2. Its code and data are those of 0.1.0, but for the
dataset card that its Harbor export writes, whose example config for platoon no longer sets image_map, which
platoon 0.3.0 removed. The release for platoon 0.2 stays 0.1.0: there, keep the extension within 0.1
(platoon-textcraft>=0.1,<0.2), since platoon 0.2 refuses the components of 0.2.0, also when it is installed without
its extra. It is a minor release, not a patch release, because it moves the platoon it installs to a new minor
release.

Install: pip install -U 'platoon[tinker,ors]>=0.3,<0.4' 'platoon-textcraft[platoon]' (the environments alone, for any
ORS client: pip install platoon-textcraft).
Full notes: CHANGELOG.md and the
extension's README.

platoon 0.2.1

Choose a tag to compare

@ApGa ApGa released this 19 Oct 13:12
5cce017

Fixes

  • A busy node no longer makes the trainer kill rollout workers that were already forked. On a node whose CPUs are
    all busy, the trainer's event loop could be held up for longer than the fork server's one-minute deadline. When it
    ran again, it gave up on workers the fork server had forked meanwhile, killed them and started them again by exec,
    each importing the worker's stack itself, which loaded the node further. The trainer now reads what the fork server
    sent before it judges a deadline, so an answer, or the hello of a server that finished importing, is used however
    late it is read, and a fork fails only once the server has answered no request for a minute (one that answers the
    requests before it is working through them, in order). A fork server still importing the stack after the worker's
    startup budget is no longer killed and replaced by another that imports it from the start while the workers wait for
    that one: they start by exec meanwhile, and the server forks the next ones once it has imported the stack. If it
    then finds that it cannot serve (a module did not import, a thread runs in it), the trainer logs why and starts no
    other, as for a server that says so in time.

Install: pip install 'platoon[tinker,ors]>=0.2,<0.3' (0.2.1 is a bug-fix release; a range within 0.2 picks it up).
Full notes: CHANGELOG.md.

platoon 0.2.0

Choose a tag to compare

@ApGa ApGa released this 19 Oct 12:20
dd6f34b

Platoon 0.2.0 is the first release of platoon on PyPI. It replaces the original platoon: platoon now trains agents
with reinforcement learning through a Tinker-compatible training service, runs any agent harness, and serves
environments as services. The original AReaL-based trainer, its standalone inference scripts and its plugin layout
are gone, and there is no step-by-step upgrade guide.

  • The original code is at the tag archive/v0.1. To install
    it from git, pin that tag: pip install 'git+https://github.com/ApGa/platoon@archive/v0.1'.
  • The RAO paper's code: the experiments of
    Recursive Agent Optimization (RAO) ran on the code of the
    apga/rao-snapshot branch, which stays.
    recipes/rao carries the paper's TextCraft-Synth
    experiments over to Tinker: its single agent, its recursive agent, whose subagents are rewarded for crafting their
    own targets (the paper's Dense rewards), and its Sparse ablation, in which every subagent takes the root's success.
  • Why 0.2.0: the original platoon gave its version as 0.1.0, so the two never share one.

Install

Python 3.12 or later (CI tests 3.12, 3.13 and 3.14), on Linux, in a new virtual environment:

pip install 'platoon[tinker,ors]>=0.2,<0.3'
uv add 'platoon[tinker,ors]>=0.2,<0.3'      # in a uv project

Add hpc, as in platoon[tinker,ors,hpc], for Singularity/Apptainer or enroot sandboxes on clusters.

Extra Installs Needed for
tinker the Tinker SDK (0.16.1) and tinker-cookbook (0.3) training and evaluation
ors the openreward SDK ORS environments, including the examples
hpc hpc-sandbox (0.3), from PyPI Singularity/Apptainer and enroot task sandboxes, for Harbor datasets on clusters
  • Keep the range when you pin platoon. Below 1.0, a minor release (0.3.0) may change the Python API, the config
    format or the command line, and a patch release (0.2.1) only fixes bugs. The range also makes pip replace an
    original platoon 0.1.0 that is already installed.
  • A project that trains with platoon names it as a dependency with the same range:
    Depend on platoon from a project.
  • torch comes from PyPI as its CUDA build, with several GB of CUDA libraries that platoon, which runs on the CPU,
    does not use. uv takes the CPU build instead: uv pip install --torch-backend cpu 'platoon[tinker,ors]>=0.2,<0.3'.
  • TextCraft is the first-party extension platoon-textcraft 0.1.0, on PyPI from its own tag:
    pip install 'platoon-textcraft[platoon]'. It also exports TextCraft-Synth as a Harbor dataset.
  • The tutorials, examples and recipes are in the repository, not in the package. Clone it at the tag:
    git clone https://github.com/ApGa/platoon.git && cd platoon && git checkout v0.2.0 && uv sync --extra tinker --extra ors.
    Number search, the first run's environment, ships in the package as the built-in named environment
    platoon-number-search@1.

What it does

Training

  • Reinforcement learning through any Tinker-compatible training service, hosted Tinker or a self-hosted server
    such as miles-tinker. platoon train runs groups of episodes per task,
    builds training data from the exact sampled tokens and centers rewards within each group (mean or leave-one-out
    baselines, with depth weighting for recursive agents).
  • Every training task trains. Tasks are drawn in seeded shuffled passes that go on from one epoch to the next, so
    no task is left out of every epoch. Several environments entries make a mixture, whose tasks training interleaves
    in a weighted fair order: weight: tasks, the default, draws every task of the mixture equally often.
    platoon train logs each entry's share, and platoon config validate --tasks prints the same before a run.
  • Metrics by label. train.label_by names the dataset column that labels the tasks, by default the mixture
    entry. Each step's metrics and each evaluation's break down by its labels, and group limits and straggler
    policies key on them.
  • Evaluation that rolls out as training does. A training config's eval.workflow_config is laid over
    train.workflow_config: an evaluation during training keeps training's rollout and rollout-worker settings unless
    it sets them.
  • Cadences in training steps, concurrency in rollouts. eval.every and checkpoint.every count training steps,
    and 0, the default of both, does neither. The first evaluation is at step every; eval.at_start: true also
    evaluates the starting policy, at step 0, and a run that resumes does not evaluate a step twice.
    max_concurrent_rollouts counts the rollouts that run at once. A training config without an eval section does
    not evaluate.
  • One run per run directory. A run records its model, renderer and held-out tasks (identity.json), and a
    resume that would change them stops before it opens the training service.
  • Runs that start again by themselves. With watchdog.max_restarts: N, platoon train starts again from the
    newest checkpoint after a failure, and reaps the sandboxes its run left. platoon sandbox reap RUN_DIR reaps those
    of a run that was killed.
  • A missing or refused Tinker key is one line. platoon train, eval and rollout check TINKER_API_KEY
    before they prepare, write or reap anything, and a key that is missing, malformed or refused by the service ends
    them with one line and exit 78 instead of a traceback.
  • Recovery checkpoints that do not fill the disk. With checkpoint.keep_recovery: N, training deletes its
    recovery checkpoints beyond the newest N: never a permanent one, the one a resume starts from, or one that
    rollouts may still sample from.
  • Outcomes that say nothing about the policy (a crashed grader, a dead sandbox, a timeout) never train as a
    reward of 0.
  • Subagents. An agent starts subagents with ctx.agents.spawn, under step budgets and depth limits. One rollout
    trains every agent in the tree, and episode programs (@pt.program) compose episodes and shape rewards.

Agents, models and harnesses

  • Any harness: a Python function (@pt.harness) or a command-line agent described by a TOML spec, whose
    options a config sets for each environment (harness_options). Built in: platoon-codeact, platoon-react,
    platoon-bash, mini-swe-agent and openhands, and platoon-oracle and platoon-noop, which run a task's
    reference solution or nothing, to check a dataset before training.
  • Harnesses inside task sandboxes, also sandboxes without a network, from toolchains built once per site.
  • A recording model gateway. Every model call, from Python (ctx.chat) or from any client over
    OpenAI-compatible HTTP, is rendered, sampled, parsed and recorded with its exact tokens, and an MCP tool plane
    serves each agent the environment's tools. The OpenAI Responses and Anthropic Messages dialects are experimental.
  • The sampling timeout in the config, beside the model: train.sample_timeout_seconds and
    inference.sample_timeout_seconds. Without them, TINKER_SAMPLE_TIMEOUT_SECONDS sets it, else 600 s.
  • Tool calls for any chat template. The chat_template renderer renders with the model's own template and reads
    tool calls with a parser chosen by name, found by a round trip through the template (tool_call_parser: auto), or
    written by you.
  • Evaluation with hosted models. platoon eval and platoon rollout evaluate a model behind any server that
    speaks the harness's own dialect (inference.hosted), and the key never reaches the agent's sandbox.
  • A directory for each evaluation. platoon eval writes each start to a new directory,
    runs/eval/<policy>/<UTC time> (under the site profile's runs when it sets one), unless inference.output_dir
    names one, and never resumes another policy's rollouts.

Environments and sandboxes

  • Environments as services: OpenReward (ORS) environment classes, in process or on environment servers; Harbor
    task directories, graded by their own tests in process, Docker, Singularity/Apptainer or enroot sandboxes; step
    environments through pt.gym; and task tables.
  • Harbor datasets as published. A Harbor environment names a dataset (hf://org/name@commit, or a local
    directory) that platoon harbor fetch downloads once and pins with a lock, and platoon harbor images builds its
    tasks' images once per site, also from Dockerfiles that ADD a URL or a git repository, which the oci builder
    builds with BuildKit as Docker does. holdout holds out whole groups of tasks for evaluation, so that no held-out
    task holds data that training has seen. The README's
    Train on a Harbor dataset goes
    from a site profile to training with configuration alone, and
    examples/harbor-dataset is that project,
    ready to copy.
  • One command prepares a config. platoon prepare CONFIG makes, once per site, what the config's runs need:
    datasets, images, toolchains and the tokenizer. A run whose toolchains or images are missing stops before any
    rollout and names the command.
  • Harbor grading that agents cannot forge. Grading kills every process the agents left running, makes the tests'
    and the reward's directories afresh, and reads only regular reward files.
  • Rollout deadlines from the task. Without a rollout_config.timeout, a rollout's deadline follows from the
    harness's own time limit and a Harbor task's `...
Read more

platoon-textcraft 0.1.0

Choose a tag to compare

@ApGa ApGa released this 19 Oct 12:20
dd6f34b

platoon-textcraft 0.1.0 is platoon's first-party TextCraft extension. It is on PyPI from this tag, built from the
same commit as platoon 0.2.0.

  • TextCraft, from ADaPT, and TextCraft-Synth, platoon's synthetic variant
    with deeper crafting trees, as OpenReward (ORS) environments. TextCraft-Synth's train-medium split holds the 852
    tasks that the RAO recipe trains on.
  • The named environments textcraft@1 and textcraft-synth@1 for platoon, with the CodeAct prompts of
    platoon's TextCraft agents.
  • An export of TextCraft-Synth as a Harbor dataset.
pip install 'platoon-textcraft[platoon]'   # with platoon 0.2 (platoon[ors]>=0.2,<0.3)
pip install platoon-textcraft              # the environments alone, for any ORS client

The extension's README
describes its environments, its named environments and the Harbor export.