Skip to content

entrust 0.20.0: experiments, an advisor and a swarm

Choose a tag to compare

@Nowely Nowely released this 18 Sep 07:42
· 89 commits to main since this release

Added

  • A fourth skill, /entrust:experiment: one registered experiment on the orchestrator's own rules. A
    protocol before any agent (a hypothesis that can be false, arms with the comparator the question calls
    for and one "go" for all of them, material frozen with its ground truth before any arm sees it, the
    research ruler's metrics counted per outcome with n and an interval, a judge that reads the returns with
    their first line removed and the arms lettered, a stop rule and budget), each arm an orchestrated run
    with its own directory and the same brief, a failed arm never re-run to a better number, the
    orchestrator's conclusion that states no cause the design cannot carry, then the user's verdict that
    decides, and neither rewritten afterwards. The record is experiments/<date>-<slug>/ under the state
    directory, beside the orchestrate runs, written only by skills/experiment/scripts/experiment.mjs
    (init, arm, add, export, list; every write create-only, the record closed by its verdict, a
    records root that is a symbolic link refused, an export destination under the state directory refused),
    because a coordinator's own write there is refused as a sensitive file (measured 2026-09-08) while a
    script handed the path is not; export copies a record unchanged into a checkout as
    research/<date>-<slug>/, the layout the repository already keeps. Cleanup neither lists nor removes a
    record. Five protocols are registered in references/protocols.md: E1 one Sol against thirty-four Luna,
    E2 the cheap-first cascade, E3 the standing advisor against per-call advice, E4 swarm coordination, E5 a
    mixed team on one deep task. Pinned by evals/experiment.test.mjs: page cases by the words that carry
    each rule, a negative case against a sentence that would let a record change, and nine script cases in
    a scratch state directory. Why: the 2026-09-17 research round produced fifteen hypotheses and a rule
    that only matched runs with an independent judge measure improvement; without a vehicle they stay opinions.

  • A fifth skill, /entrust:advisor, prompt only: one standing top-row advisor of the other model family
    for one run, named in the plan with its turns, asked one question at each decision point (the split,
    the composition, a verdict about to be adopted, a stall) with the coordinator's own decision in the
    question; it returns a recommendation with reasons, one alternative and what would change its mind,
    never implements, never judges its own advice, holds a slot only while a turn of its runs, and every
    decision point is written down before and after in a notes file the synthesis names. The page states
    no benefit until protocol E3 has run. Pinned by evals/advisor.test.mjs, including a case against any
    sentence that would hand the advisor a role it must never take. Why: the owner asked for a standing
    advisor by a separate command to test its effectiveness; the 2026-09-17 research found no source that
    had measured one.

  • A sixth skill, /entrust:swarm: up to fifty bulk agents over a file of units, each made and run by
    skills/swarm/scripts/swarm.mjs through the sibling launcher as agent <id> at <run>/<id>/report.json
    with its agent/ beside it, the layout the cleanup lists as a run; at most --concurrency at once,
    fifty enforced by the script; a summary written outside the run, in an agent-scratch directory, from the
    launcher's status lines, so a stale report under a taken path is never this run's outcome; a signal
    stops further launches and reaches every running agent. A Terra swarm counts as the bulk row does,
    against the swarm's own cap, stated on the page as this mode's one override; every brief is a read agent
    on Luna or Terra at the effort the orchestrate page sets. Sharing nothing is the default; shared state
    (a queue claimed by mkdir) and free messaging are protocol E4's arms, and peer messaging as
    verification stays on the research's do-not-adopt list. Pinned by evals/swarm.test.mjs: page cases
    by the words that carry each rule, a case that no strong or top model is admitted, and swarms against
    the fake app server for the layout, the dollar-quote substitution, serial concurrency, a stale report
    and a signal. The roles reference gains the standing advisor and the swarm reducer, and the orchestrate
    suite's roles case pins that neither may write on a tree. Why: the owner asked for a swarm mode that
    offloads the orchestrator's launches and waits, and for both coordination shapes to be tried as
    experiments rather than decreed.

Changed

  • The caps count turns in progress: separate advisor, critic and architect threads may take turns within
    them, and a thread waiting for another message uses no slot; fable is for Fable agents within the agreed
    cap. A plan may propose a cap of its own with its reason, and the user's word sets it for the run. Why: the
    owner's reading of the pool on 2026-09-17, after a design that counted idle threads against the cap.
  • The completeness critic reads the user's request, the final answer and its evidence before every final
    answer of an orchestrated run, one fresh strong-row reader named in the plan, and returns done, partial or
    not done with what is missing, unverified or unread; a publication is read the same way; a one-agent task
    has no judgement agent beyond it. Why: the owner's corrections at the synthesis stage were 8 of 27 in the
    record, and a README once published an inference from absence unchecked (T1-49).
  • A roles reference, skills/orchestrate/references/roles.md, linked from the bounds paragraph: 19 roles
    with what each does, may write and returns, when it is spawned, its tier and the runs that used it; no role
    is a phase of one piece of work. Why: the role set was in practice the tier table's four rows; the
    2026-09-17 survey found role catalogues in ten of thirteen frameworks and the record twelve roles assigned
    without a table.
  • The orchestrate page gains eight rules from the 2026-09-17 research round
    (research/2026-09-17-orchestration-practices/: 185 survey claims mapped to the two pages and 54
    coordinator incidents from the local record; the T1 ids below are that round's, and 426:973 is a
    line of the session transcript T1 cites). Each rule and the defect that paid for it:
    • A decisive check runs before any panel is commissioned; dependent execution stays in one agent and
      its verification stays independent. Why: two naming rounds put sixteen agents on proposals, reviews
      and a verdict before the check that decided had run, and the second round's winner fell to a
      collision check after the verdict (T1-45; 426:973, 426:1208), while the two tasks the coordinator
      kept in its own hands landed with critics only (T1-19, T1-48); the controlled comparisons say the
      same of sequential and tool-heavy work (S2-02, S2-03). The rule orders the check; the fresh verifier
      of "you never grade your own work" stays, and a new eval case fails if the page ever says otherwise.
    • A judge's verdict that lacks its decisive check is unknown in result, the missing check named in
      open; a Codex judge's check goes under the sibling's EXPECT: rule. Why: judge Astra J3's
      proofbound was disqualified by a collision check no agent had run (426:1208); judge J2's "keep"
      was reaffirmed on new evidence (426:1005) and then set aside by the owner's rule that the plugin
      name carries no vendor (426:1018) — one verdict of six fell to a check, not the two that T1 §2.4
      counts; T1-37's own row says the conclusion held.
    • Between selection rounds the coordinator records what was rejected, what was learned and what
      still blocks; two rounds on the same blocker are a stall and become a new plan for the word. Repair
      rounds keep their ladder: two rounds, the top row, then the user. Why: the naming rounds each ended
      on the same blocker and the two-round rule, which counts fix rounds, never tripped (T1-45).
    • Before the plan is shown, each agent's required commands are checked against its planned rights and
      environment; uncertain prerequisites are probed cheaply and unmet ones go into the plan. Why: a Sol
      review turn spent 2,649,693 tokens and left its collision check unrun because SSL failed in its
      sandbox, and the coordinator redid it (T1-44; 426:1005); ten of thirteen agents in one run ended at
      exit 6 on declined requests (T1-51). Nothing is reassigned after a refusal: the sibling's rule
      against widening rights on a refusal stands.
    • One assembled brief is opened whole before any fan-out, its input paths checked in the agent's
      planned tree, its item count and each quoted claim against its source. Why: the split critic reads
      the decomposition, not the file the generator wrote, and three generator defects each reached every
      agent — a join that paired all twelve reports with the wrong paragraph (T1-05, 2026-09-11, the one
      outcome-blocked incident of the coordinator's own), a doubled path segment in all twenty prompts
      (T1-09, 2026-09-12) and a quoting slip that gave each of three agents one set of four (T1-39,
      2026-09-17); a path check alone would have caught only the second.
    • A refuter's finding is one that changes correctness or a stated requirement; the rest goes to
      open. A verifier's brief names its target and whole scope, and its return separates what it
      checked from what it did not. Why: round 08 of the markup round made twenty-seven edits against a
      wave's eighty-seven findings and five owner decisions and brought ten regressions of its own (T1-50;
      research/2026-09-11-markup-round-0/rounds.md); the coordinator once bounded a report clipped at
      50,000 characters to the parts it had read (T1-30; 426:33), where three earlier claims from partial
      evidence were cut by agents (T1-08, T1-12, T1-49).
    • Every alternative in the plan is numbered with its cost, the recommendation marked, and the plan
      says what "go" selects. Why: a bare "го" was read as assent over forks the plan had left open three
      times (T1-23; 30a:204 on 2026-09-12, 30a:848 on 2026-09-13, 426:92 on 2026-09-16; no objection
      followed) and a wait was offered as free that was not (T1-38; 426:1214, 2026-09-17).
    • The bulk row's count is derived from the units and the plan says why that many; the plan states
      expected tokens by tier and role from comparable runs, unknown where unmeasured. Why: eighty
      agents were launched on the word "bigger" against a page that already said six alive, and twelve
      findings survived (T1-01, T1-03); the user stopped a wave of fifty-one for its cost before any cap
      existed (T1-53); an Astra ran at 1 % quota with the only reproduction path on it (T1-07). No
      number sits on the page: the record's pooled medians mix roles and task shapes (Luna 13.6 k for a
      recognition read, 742 k for a tree verification; Astra 585,186 over six agents under the
      conventional median, where T1 §2.4's 903,705 is the upper-middle value), and Claude agents' tokens
      are unrecorded (T1-U3).
    • The completeness critic is one fresh strong-row reader chosen by the agreed composition and named
      in the plan, given the whole publication and its evidence once, before it goes out, never per
      return. Why: no run of seven had spawned one (T1 §4) and a research README published an inference
      from absence unchecked (T1-49).
  • "Prefer Luna to Haiku in the bulk row" drops "and smarter, and four times cheaper". Why: no Haiku token count exists
    anywhere in the record (T1-U3), so the price half was a claim, not a measurement, and nothing in the record measures "smarter". The preference
    stays as the owner's: four of fifty Haiku returns were lost to the schema and, of about thirty-six
    candidate findings from sixty-six cheap extractors, twenty-four were refuted (T1-02, T1-03); twenty
    of twenty Luna located the file under a broken path (T1 §2.3a) and thirteen of thirteen agreed with
    a judge who read the tree (T1-26). Two unmatched cohorts, not a comparison; the matched one is
    phase 3's.
  • The page's line budget in evals/orchestrate.test.mjs moves from 155 to 156 for a page of 156
    lines; ten cases pin the new rules by the words that carry them (B7, C9–C11, D9, D10, E7, F7, F8,
    G7), the three verification bullets that changed carry their new text in F2, and B7 fails if the
    page ever tells the orchestrator to grade its own work.

Verified for this release

Check Result
offline suites (npm test) 15 of 16 green: orchestrate 59, advisor 10, package 11, agent-contract 14, agent-run 15, swarm 16, attach-pasted 10, experiment 20, cleanup 42, worktree 30, cli 111, conformance 89, lock 58, protocol 147, fidelity 15
syntax, every shipped script and suite 26 files parse under the declared Node floor's successor (see below)
the record mechanism, live on one machine experiments/2026-09-18-probe/ made by experiment.mjs under the real data directory, where a coordinator's own write is refused

Measured on: Node v24.11.0 (the floor package.json declares is 22), codex-cli 0.153.4 (the pinned build), Claude Code 2.1.170, macOS.

Known issues

  • The two live gates were not run. fidelity ran offline and reports "live turn: NOT RUN"; orchestrate-live is the one suite npm test did not run. The Codex quota was exhausted at release time and the owner chose to release without them, so nothing in this release is measured against a live server or a live orchestrated session. The --report-file delivery assertions and the orchestrate mode's five live cases stand on the previous release's runs.
  • The five registered protocols are registered, not run. E1 to E5 in skills/experiment/references/protocols.md have no results; the advisor and swarm pages state no benefit, and say so.
  • Two defects stay recorded in the repository's ISSUES.md, neither fixed here.